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STATISTICAL INFERENCES ABOUT TRUE SCORES* 


FREDERIC M. Lord 


EDUCATIONAL TESTING SERVICE 


Formulas are derived for unbiased sample estimators of any raw or 
central moment of the frequency distribution of true scores. A general method 
is developed for obtaining from each examinee's observed score a least squares 
estimate of his true score. 


The ordinary “observed” test score frequently can be used in practice 
without thought of separating it into what must logically be its component 
parts: true score and error of measurement. From a scientific point of view, 
however, the entire concern must be with true score; the observed score 
can be of scientific interest only as it leads to inferences and generalizations 
regarding true scores. Some new methods for making such generalizations 
are reported in the present article under two separate headings: “Inferring 
the Shape of the Frequency Distribution of True Scores” and “Estimating 
the True Score of Each Examinee.” The methods given here for inferring 
the true-score moments are based on different mathematical models than 
that used in a previous derivation [10]; the formulas presented here are 
considerably easier to apply in actual practice. 

The matrix-sampling model, used in all but the first two sections of 
this article, readily yields a wide variety of basic results of importance to 
test theory. The shape of the bivariate frequency distribution of observed 
scores on two parallel forms of the same test, for example, can be predicted 
from the data provided by either form alone. Again, the relationship between 
true scores on two different tests can be estimated from observed data; in 
particular, the question can be considered whether the true scores have a 
perfect curvilinear relationship. Although it is easy to obtain estimators for 
these and other purposes, it is more difficult to obtain usable formulas for 
their sampling errors, at least in the case of those estimators that involve 
terms above the second degree. (Some standard error formulas for second- 
degree estimators are given in [9].) Since a knowledge of the sampling errors 
of the estimators is important, it is hoped to obtain formulas for more of 
these before further estimation equations are published. 

In the present article, formulas are derived (see Rule 1) for unbiased 
sample estimators of any raw or central moment of the frequency distribution 
of true scores. Explicit and relatively convenient computational formulas 
are given for both raw and central moments up to and including the fourth 
езеді research was carried out under contract Nonr-2214(00) with the Office of Naval 

rch, Department of the Navy. 
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order. Methods for fitting frequency curves to a set of estimated true-score 
moments are briefly discussed. 

А general method is developed for obtaining from each examince’s 
observed score a least squares estimate of his true score. Detailed com- 
putational formulas for making such estimates are given for the case where 
the regression of true score on observed score is assumed to be linear and for 
the case where this regression is assumed to be a third-degree parabola. The 
method can be extended to the case where this regression may be approxi- 
mated by a parabola of any specified degree. 

Inferring the Shape of the Frequency Distribution of True Scores 


The true score of an individual is customaril 


у estimated by the linear 
regression equation ([7], eq. 11:20) 


(1) E = E F Talta — 2), 


The Assumption of № ormally Distributed Errors 


‚ А first approximation 
be obtained by assumin 


distribution. Since e i » Vol. 1, ch. 4) of the true-score 
definition, x, = 


› its ef. is exp (— 1252), By 
=e+é, E А well-known theorem states that the ak ы. sum 


c.f’s. The c.f. of x is thus equal to the product of their 


(2) 90 = $«0 exp (32.3), 


and the cumulant generating function for т is 
(3) Уй) = log (ù 
= и) - peo. 
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Since the rth cumulant of the distribution of x is the coefficient of (zt)"/r! 
in the Taylor expansion of 4. (0), it is clear that the true-score distribution and 
the observed-score distribution have identical cumulants except for the 
second; the variance (second cumulant) of the true-score distribution is 
equal to the variance of the observed-score distribution minus the variance 
of the errors of measurement, as is, of course, well known from other lines of 
reasoning. 

Given the moments of the observed-score distribution and an experi- 
mentally determined estimate of the standard error of measurement, estimates 
of all the moments of the true-score distribution can thus be readily written 
down from standard formulas relating moments to cumulants ([8], eqs. 
3.30 and 3.34). (The shape of a distribution is ordinarily determined by its 
moments. Methods for doing this will be discussed briefly at a later point.) 

The foregoing convenient method for determining the shape of the 
true-score distribution may be a good approximation in the usual situation 
where the observed-score distribution is nearly normal. However, in this 
case the problem has little special interest, since the true-score distribution 
will then also be approximately normal with known mean and variance. 

Actually, it is clear that when an examinee’s true score is close to zero 
or to n, the number of items in the test, the error of measurement in his 
observed score is likely to have a distribution that is skewed and a variance 
smaller than that for an examinee with a less extreme true score. Some more 
satisfactory assumption about the nature and frequency distribution of the 
errors of measurement is needed. . 


The Matriz-Sampling Model 


The matrix-sampling model ([9], sec. 8) completely abandons the assump- 
tion that the errors of measurement are distributed normally and independ- 
ently of true score. Instead, the group of examinees tested is considered as 
drawn at random from a very large population of examinees (“буре-1” samp- 
ling) and the test is considered as a sample of items drawn at random from a 
very large pool of items (‘‘type-2” sampling). The error of measurement in each 
test score is, by definition, the sampling fluctuation arising from the type-2 
sampling process; its frequency distribution and other properties are then 
inferred by means of standard sampling theory. Methods for estimating true 
scores are then readily devised. y 

Tt is assumed that the score on the test is either the number of items 
answered correctly, і.е. т, = 21.1 а) Where та = 1 if examinee a answers 
item 2 correctly, and 24 = 0 otherwise; or, more conveniently, the proportion 
of items answered correctly, to be denoted by 
(4) „езен 
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Unless otherwise specified, the "score" of a test will hereafter refer to the 
proportion-correct rather than to the mumber-correct. 

For convenience, this notation suppresses a third subscript, и, which 
must always be understood: Tuia 15 the score of examinee a on the ith item 
in test и. Note that the subscript 7 identifies а unique item only when a 
specified test (sample of items) is under consideration; otherwise, the sub- 
script 2 merely denotes the ith position in any test и. In general there will be 
only one test for which real data are actually available; all other tests are 
hypothetical samples of items drawn at random from a hypothesized item 
pool. 

The symbol Е, will denote the operation of taking the expected. value 
of a statistic in type-2 sampling, i.e., the operation of taking the average 
value of the statistic over all possible samples of n items each. Ti 


or example, 
E,2;, is the equivalent of 


à 1 U 

lim — я 

lim 7; 5 Suis 

Thus the symbol Е can be manipulated in the same way as 
summation sign. (Note in particular th: 


variables is equal to the sum of the expected values of the variables, and 
that the expected value of a product of uncorrelated variables is equal to the 
product of the expected values of the variables.) 


The expected value over all examinees 
by Е, . The expected value taken simultaneously over all possible combi- 
nations of n-item tests and over all examinees in the population will be 
denoted by Æ,» . The average value over all examinees in the group (sample 

A, (or A,) when the number, №, of 
examinees is very large and by the equivalent operator N~! БА UM оен 
operators E, and A, is identical, but 
it seems wise to use different symbols for the two different operations, | 
ain estimates of the moments of the 
d in type-2 sampling. For conve. 
stimates of the moments of t| 
timate is one w 


ап ordinary 
at the expected value of a sum of 


in the population will be denoted 


nience, 
he true 


у obtained by m 
Hooke [5, 6], and Robson [13]. The present derivations, however, will be 
carried through by means о 5 апа terminology more familiar to 
the average reader. 


Some further notation may be defined at this point. 


True proportion-score of ex 


aminee а (his mean proportion 
possible tests): 


-5соге over all 


FREDERIC M. LORD 5 


(5) ta = Biz, = Esa. 


Difficulty of item & (proportion of examinees answering item 7 correctly), 
for the population of examinees and for the sample, respectively: 


(6) m; = Еш, Pi = Ах, = EA 2 Via + 
N ia 
Average true proportion-score = average item difficulty: 
(7) {=т= Et = Em: = B Eza = Е.В. = Ей. 
Proportion of examinees answering correetly all of the т items й, iz, 7,9: 
(8) пиве = Ех Ха "77 оэ 
Багы. = Жата на 77 Lira + 


0, h, i, j will be used as alternative subscripts to designate items; а and b 
will be used as subscripts to designate examinees. The symbol n'" will denote 
the product n(n — 1)n — 2) =- (%—7 + 1). 
Type-2-Unbiased Estimates of Raw True-Score Moments. Now consider, 
for example, the quantity 
n(n—1) (n-2) 


PPP 


gairi 


(The product above the summation signs indicates the number of terms in 
the summation.) The expected value of this quantity in type-2 sampling is 


1 А 
(9) Е. ыы p » i Ути: = л?! 22 > У ESL, ХЕ лысы) 
т "T әжімі 
Е.Е, ivX jv 
En 
Qa p) 0) 


1 


І 


= шш. 
are as follows. (i) The E. is moved to 


'The steps in the foregoing reduction ] 
the right and the definitions of т, and тұ; are substituted for these two symbols. 


(ii) The value following the summation signs is the same for any 9, ?, and ў; 
since there are n(n — 1)(n — 2) terms under the summation signs, everything 
preceding Е, cancels. The E, symbols are moved to the left (the population 


of examinees is assumed to be so large that terms with a = b may be ignored). 


(iii) If а person's score on the ith item on а particular test (random sample 


of items) is higher than on other tests, this js no reason why his score (or 
another examinee’s score) on the jth item should be higher than on other 
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tests; hence z; and t; (and also z,,) fluctuate independently in typo-2 
sampling, and the expectation of their product is the product of their ex- 
pectations. (iv) E, is moved to the right. (v) и/ is here defined as the rth 
raw moment (moment about the origin) of the true-score distribution. 

The quantity following E; on the left side of (9) is seen to be an unbiased 
estimate of u[u . An unbiased estimate of any product of true-score moments 
about the origin may be obtained similarly by Rule 1. 

Коп 1. То get a type-2-unbiased estimate of the product yu! ui, 55 шу of 
таш true-score moments, (i) replace each ш. (ш = 1,2, ... , W) by the symbol 
т with т. different subscripts, (ii) take the r-luple sum of 
of ms over all r = >)” r, subscripts, 
value, (iii) divide by п". 


For example, an unbiased estimate for ш = (in type-2 sampling is 


the resulting product 
no two of which may assume the same 


(10) Este = т), 


where Est, is read “а type-2-unbiased estimate of.” Similarly, 


A 1 
(11) Est; ш = Е E >; Жү» 
n ini 
H 
(12) Est, ш? = MI 25. тт; , 


2 3 1 
(13) Est, мийш? = е XE. ^. ЧАС 
т DUREE тт, . 


If the number, N, of examinees in the group 


symbol тіп Rule 1 and in equations (9) through 
to obtain type-2-unbiased esti 


products of true-score moments, 
is not large, it is desirable to find type-1-unbiased е 
of z's. Such estimates are matrix-unbiased estim: 
Score moments, i.e., simultaneously type-1- 
discussion of such estimates, the assumption t 


Stimates of these products 
ates of the products of true- 
and type-2-unbiased. In the 


: hat N i i 
First of all, the two quantities tss айе dropped. 
DX 
DO = = Та, 
(14) ME 
1 N 
Da = N 25 Lialia у 


are sample means т type-1 samples and 


у hence аге type- 
of т; and z;; , respectively: 


1-unbiased estimates 
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Est, т; = pi, 
Est; п; = Di; + 


(15) 


To take a less simple case, consider the covariance between items $ and j, 


(16) $; = pu — РР; - 
According to the usual formula for the expected value of a sample covariance, 
(17) Ең; = (№ — Пе/М, 
where 
(18) Ci; = T — тт. 
Thus it is seen that 
(19) FE, 444 (Ур — pi) = в(- — “Р ры) 

N-—1 N I 

= —о; + ти = тт; . 


Now, since the sample mean of the observed test score is 
1 
(20) 2-1 Ор» 


1 


it follows from (10), (15), (14), and (20), that 

1 jl em 
(21) Est, в! = Esti = > тоо È p: = 2. 
Similarly, from (11) and (15), 


1 = 
(22) Estis a = Est: cu У ути = са D 5и. 


Likewise, from (12) and (19) 


1 
(23) Est, af = Est aj 22 22 vi 


i 


= d T »»» (рр; — р). 
n ixi 


Equations (21), 22), and (23) provide estimators of pf, ui, and pf? that are 


i in matrix sampling. қ d Р 
вы estimates that ате type-2-unbiased will differ from estimates 
Д 


that are matrix-unbiased at most by quantities of order 1 /N А Since such 
quantities will be negligible if N js at all large, further consideration will 


center on type-2-unbiased estimates. 
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Computational Formulas for Type-2-Unbiased Estimates. Although the 
type-2-unbiased estimators so far discussed are exact, these estimators are not 
yet expressed in a convenient computational form. More convenient formu- 
lations will now be obtained, retaining quantities of order 1/n, since these will 
be needed for some purposes, but in certain cases neglecting quantities 
O(1/n?). The reason for retaining quantities O(1/n) is apparent from the fact 
that the variance of the observed scores differs from the variance of the true 
scores only by quantities of this order. 

Very large М is again assumed. Further notation will be needed. 
rth raw moment of observed proportion-scores: 


1 Я 
(24) т! = N D» s , 
Variance of item difficulties: 
1 ж. 2 
(25) s= Ур в. 


т 


rth raw moment of item difficulties: 


(20) м=р. 
ШП 

The foregoing quantities are all statistics for the s 

and are thus computable from the data, 

First of all, z, has a binomial distribution in type-2 sampling. The 
factorial moments (18], рр. 56-60) about the origin of a binomial distribution 
have a particularly simple form ([8], p. 118): the rth factorial sampling 
moment of z, = nz, is 


(27) 
It is thus clear that 


ample of examinees tested 


uin) = Eg = т" 


а. 


(28) Est, и! = 1 panal Ах 

т“ gees 
The quantity on the right is a factorial moment readily computable from 
the frequency distribution of observed scores, Computing formulas for 


r = 1, +++ , 6 are given in (35) for convenient reference. Equation (: 
" - Equation (2 
be stated as a theorem. à шайы 


THEOREM 1. The rth raw factorial moment 0} the distribution of number- 
correct observed scores, x, , divided by n"! isa lype-2-unbiased estimate of the 
rth raw ordinary moment of the distribution of proporlion-correct true scores, © 

254" 


Computational formulas for unbiased estimat 


А i es of various products 
of true-score moments will now be obtained. First о 


f all, by Rule 1, 


' 
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3 Е 1 l 
(29) Este ш ХХ Бал, = стн О 
п бата exe Mie " 


where (1') denotes an rth order unitary symmetric function ([1], p. 431). 
This symmetric function is readily rewritten in terms of power-sums with 
the help of the tables of coefficients provided in [2]. The rth power-sum of т; 
is n times the rth raw moment of the distribution of the т, , for which may 
be substituted the computed value of M; , the rth raw moment of the dis- 
tribution of the p; . The results are given forr — 2,3, 4 in (36). The necessary 
M’s are readily computed for any given set of item data from the frequency 
distribution of the » observed item difficulties, p; . 

A similar but more complieated procedure may be applied to estimate 
other produets of true-score moments. For example, starting from Rule 1, 


1 n(n—1) (1-2) 


Est, шщ = - 22 > У тат: 
n 


селі 


M 


LÈ S Ужин —@ Denes 


(30) 


ES x > тот: + 2 FR тї). 


The tables in [2] do not provide the last expression directly, but they do greatly 
facilitate what would otherwise be an awkward algebraic task. The reader 
may easily verify the algebraic identity, however, by splitting up the various 
terms in the last expression, e.£ 


YY нт = DY тим 
+2 2 У тат: F >, 2 тыл; + У тт А 


and then recombining the results. Since тә = то, (30) can be rewritten as 
shown in the first equation in (31). The process illustrated in (30) becomes 


an easy one to carry out without the use of tables if all terms O(1/n*) are 


7 я 131 is O(1/m). 
dropped. In (30), the term 2 У" wT7,/n 18 
ae 2. derived similarly are given in (31) for the expected values of 


every product of true-score moments up to and including those of the fourth 
order, excepting those covered by (38) and by Theoret 1. The last term of 
the first equation in (31) after multiplication by 1/т/ is only of order т. 
All such terms are represented in the remaining equations of (31) by the 


symbols O(1/n’). 


4 ом У Ут 


Est; шш = B 
т 


23 ў È тил, — nM? + 20M), 
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Est, иш = - ИМ? — nM) Ў È y 


п 


= MP — 4пМ{ ғ = Tm] + of ) А 
(31) 


i 
2 
Ені, uf y L ce 25 a а) 


=з Sag dh да e(l), 


i 
п? 
Est, щи = ш WE E Pan 
TM D Ern ез È È mne) +01). 


The computation of the с 


Juantities in (31) that involve summation 
Signs can be further simplified, 


after substituting p's for т'8. 
= mi. 


(82) E > Pa = У) > Ашлы = A 2,2)? 


ah E £ Dp, = > Pr > Pon = E. Pr У) А.х, 


т 5 РА ть, = п > DÀ , 


| 


where 2, = A,2,,2, is 1/N times the sum of the scores of those examinees 
who answered item h correctly. Similarly 


X % S ns an 234, 
(34) > P» ian = пт, 
T к Тал = п? Ура, р 


where 2, = Astiz? is 1/N times the sum of the squared scores of those 
examinees who answered item 7 correctly. 
Equations (35), (36), and (37) summarize for convi 


computational formulas for type-2-unbiased estimates о 


score raw moments up through the fourth order. E 
through the sixth order, are obtained from (28) an 


enient reference all 
f products of true- 
quations (35), given 
d from the standard 


FREDERIC M. LORD 11 


formulas (|8), eq. 3.18) expressing factorial moments іп terms of the ordinary 
raw moments. Equations (36) are obtained as described in connection with 
(29). Equations (37) are obtained by substituting (32), (33), (34) in (31) 
and dropping terms of order 1/т?. 


Est, uj = mi, 


I 
Est; ш = ~z (пт! — nmi), 
n^ 


1 2 
Est; ду = т (пт! — З® т: + 2nmi), 
п 
(35) Est; ш = aL. (nmi — бт + 1In*mi — бит), 
n 


Est, uj = i (пэт; — 10n*mi 
n 
+ 35n*m4 — 50n^mj + 24пті), 


Est, uj = L (пет — 15п°т 
n 


+ 85n‘mi — 225n^mj + 274n^mi — 120пт). 
Est, д? -- (Mf — nMi), 
n 


(36) Est, p’ = d (фм? — За? ММ: + 20M), 


Est, uf! = zi QAM?! — 6 MIM? + 31M? + "М.М: — nMi). 


; 2,92 
Esta шш = -} (mimi — 2n D ра, — тр), 
« 3 


Est, uui = L miam? — Мт — nmi? — ит D på, 
? т 
(37) 


1 2 Bs eos 2 
Est, pf? = RII (пт — 2n' mim; 4n Ж 24), 


L (тїт — 3] mimi — Зп? 2). 
Est: ніш = nit} @ Tu В Р i > Ре 
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[Computational Note. ТЇ computations will involve (37) as well as (35) 
and (36), only the first two terms of each polynomial on the right should be 
used from (35) and (36). Equations (37) do not retain terms O(1/n"), and it 
seems to be better here to discard all terms O(1/n^) rather than to retain 
some when others are discarded.] 

Estimating the True-Score Central Moments. For many purposes, ty pe-2- 
unbiased estimates of the truc-score central (about the mean) moments, 
rather than of the true-score raw (about the origin) moments, are desired. 
These are obtained by expressing the central moments in terms of products 
of the raw moments ([8], р. 50) and then replacing these products by their 
unbiased estimators. Thus a type-2-unbiased estimate of the true-score 
variance is 


‚ ‚ 1 annie 
Est, ор = Est; (uj — uf?) = = [Ataza — 1) — i^ Mt? + nM] 
(38) 


1 
=] 188 — al — 2) +]. 


A matrix-unbiased estimate of the true- 


score variance can be obtained 
from (22), (23), and (16) if desired: ч 


Est; e; 


Ш 


1 N 
NE М-1 ХХ (р; — Фо) 


N її а a n 
чы К-т (2 Ess- Dat Жу) 


т 


N " 
Nu Est, ої. 
It is interesting to note that if test reliability is defined by the equation 


(40) та т? 


zz 


— Est; of 
2 , 


then substitution from (38) shows that the test reliability is the same as the 
Kuder-Richardson formula-20 reliability coefficient, т, as is conveniently 
seen from ([14], eq. 27). 


Estimating the Shape of the True-Score Distribution 


Ап estimated frequency distribution of true Scores may be obtained in 
a variety of ways from the moments. If only the first four true-score moments 
have been estimated, Pearson-type curves may be fitted by the formulas 
given in [3]. If more than four moments are available, a Gram-Charlier or an 
Edgeworth series may be fitted ([8], pp. 147-156). 

The frequency distribution may be fitted to the estimated raw moments 
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that are readily obtained from (35). Better results will presumably be 
obtained, however, by fitting the distribution to the estimated central 
moments obtained by the method of the immediately preceding section. 


Estimating the True Score of Each Ехататее 

Suppose that the psychologist is continually testing many groups of 
examinees with randomly parallel forms of some test. Suppose that he makes 
ап estimate, г. , of the true score of each ехатіпее. Over a period of time he 
wishes to minimize the sum of squares of the discrepancies between his 
estimates and the true values—he wishes to minimize E,.(f. — £4). (The 
problem here is basieally similar to that diseussed by Robbins in ([12], p. 
161), but there are important differences and the method of estimation used 
here is different from his.) 


Linear Estimation of True Score from Observed Score 


In the simplest case, it is assumed that the estimate is a linear function 
of observed score. In this case, the problem is to minimize 


бұл Е,(Вг, + C= к? 


by a suitable choice of В and С, which are assumed to be functions of the 


true scores, not of the observed scores. 

Since in type-2 sampling z, is actually a sufficient estimator of f, , № 
might be thought that S; would be minimized by setting В = Тапа C = 0. 
This is not the case, however. 

The necessary derivatives are 


(41) 2S: — 2E s Be, + С — t), 
(42) 95 — эБы(Вь + C 0). 
Set (42) equal to zero, take expected values, and solve for C to obtain 
(43) С = #1 — B. 
Substitute (43) in (41) to obtain 
(44) . (Е,,2 — В = Е. — ағ. 
Ву (28), 
2 lj num p uds 
(45) Eye = 23 Fus + ai) = n H2 FS ш. 


Since Ғы. = Е.б. = ni (44) becomes 
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rus ш- ш? _ то z 5 
n 1 1 п (= о + 41 — 
(46) эшш. Беш – и + & s 


n ГЕ 1-9 =. 

n—1 (n — Der £1 — 0 

Since £, £^, and of are unknown, they must be replaced by sample esti- 
mates. These will have а type-2 sampling variance of order 1/n, and will 
therefore probably differ from the true values by an amount of order M мл. 
Such а substitution into the last expression in (46) will change the value of 
B only by O(n 7^), because of the n — 1 in the denominator of the second 
fraction. If unbiased estimates from (35), (36), and (38) are used, the esti- 
mated value of B is found to be 


__в [1-2 4 s/G — 1) 
a B = 1 Е ns; + ns;/(n — 1) | 


This quantity differs from the Kuder-Richardson formula-21 reliability 
coefficient ([4], р. 225) 


СОЕ. _ a — 2) 
(48) Ta = m=i Е a" ] 


only by quantities of order 1/27: 


n EE E 


n —1 ns; + пзу/(% — 1) 


fai 


= — Sls: — 2(1 ~ 2) 1 ) 

í (в — Оз — Ds --s]-^ o | 

Consequently, т», may be used as an estimator for B. It should be noted that 

7а and т differ by O(1/n). Hence го is not an adequate estimator of B in the 
present situation, since r; also differs from 1.00 by O(1 /n). 

The linear equation for estimating an examinee’s true score from his 

observed score is thus seen to be 


(49) % =2+8e.-2 о E = E + Ва, — 4). 
This last equation is the same as (1) except that В now appears in place of r.. 
Curvilinear Estimation of True Score from Observed Score 


A virtue of the present approach is that it can be extended to the case 
where the regression of true score on observed score is curvilinear. This case 
is likely to arise when the true-score distribution in the group tested is 
substantially different from the usual symmetrical bell-shaped form. This 
situation will arise not only whenever the group of ex: 


er ә aminees is а peculiar 
one, but also whenever the test administered is too hard or too easy for the 
group. 


чеч ————————————————  }үўйҤЧ -өзн- 
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FIGURE 1 


Regression (heavy line) of true score (2) on obtained score (z) when true score has 
a dichotomous frequency distribution 


Figure 1 presents an extreme example to illustrate why a rectilinear 
relation cannot always be expected. The situation in Figure 1 is that half of 
the group have true scores at {о and half at fi . The conditional frequency 
distribution of the observed scores for each of the two true-score values is 
indicated by a bell-shaped curve. Since the average error of measurement is 
zero, the mean of each conditional frequency distribution lies on the 45- 
degree broken line, which is the regression of observed score on true score. 
It is graphically evident from the figure that the other regression, the re- 
gression of true score on observed score, must lie very close to the heavy 
ogive-shaped line and must be very sharply curvilinear. 

If the curvilinear regression of true score on observed score can be 
approximated by a rth degree polynomial іп 2, , the equation of this curvilinear 
regression can be obtained by finding the values of the B’s that will minimize 
S, = Е.(В,-- Bite + Bos + Ва — ©) and by writing the re- 
gression equation as 
(50) Е, = Bo + Ва, + Ваг + °° + Bz, 
where the В’з are sample estimates of the B’s. 

When the partial derivative of S, with respect to each B is set equal to 
zero, there results a set of r linear equations in r unknowns which are basically 
the same as the "normal" equations used to determine curvilinear regression 
coefficients in conventional situations ([8], eq. 22.20; [11], рр. 429-431). For 
example, when r — 3, the equations found by differentiating 5; are 


| Е.б = Bot В,Е 24 + В.Е, “РЕ BE " 
= Bs + В.Е, Е B.E xs В,Е, 2. › 
BoE ze + В,Е, 22 = В.Е. E s В.Е, , 


ШҮ 


Еа = 
Et = В.Е. + В.Е = В.Е... + В,Е,.#. . 


(51) 
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In order to evaluate „г, , it is first convenient to express z; as а sum of 
the polynomials zi (7^ = 1, 2, --- , т), as in (45). The expected values of 
each of these polynomials is then readily obtained from (28). Thus, 


[2] 7 


T 1 
Ег, = 2 (өш + пш), 


1 
Exc, = ja Esa + Bal" + at", 


азы + Bn! al + na, 

(52) Е, = 4 (n'* n4 + 6n" yh тп! u, + пш), 
Еш = 5 (nu + 10n! us + оби yh + 15n'? 4. аш), 
Eye = Js (nud + 15n"! + 65n! ys 


+ 900! 4 Bin!" yr + nyt). 


The numerical coefficients on the right are the same as those obtained when 


ап ordinar y raw moment is expr essed as a su о ас 1 nts 
b m of f: t 
( | «e ) actorial поте 


Similarly, 


Erata = ш, 


1 
(58) Еа = па us + тшу), 


Y ouf 1 
Еф = aj QT ub Bn р лш). 
It is helpful to rewrite (51) by makin 
C, = n(B, — 1), С, = nB: , C, = nB,. With 
CoBra + Collate + CE, + С, = nS, — Eug) = 0 
4 7 K л ! 
(ау Cote + GERE + OB + Сй, = кш, - клу, 
C3022 + Се + С.Е, + СЕ, = "Еа 
С. + C4E sg; + С.Е, + Cle, = "Еа 


g the substitutions C, = nBo » 
a little rearranging, (51) becomes 


= E22), 
des Ese. 
The reason for preferring (54) to (51) is that three of 
(B's) in (51) are quantities O(n^*) and the fourth diffe 
quantity O(n )); in (54), all the unknowns are 0(1) 


the four unknowns 
ers from unity by & 
; Le., are functionally 
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independent of the value of n except for terms that become small as n becomes 
large. Furthermore, when (52) and (53) are substituted into the right side 
of (54) all quantities O(n) cancel leaving only quantities O(1). 

Adequate approximations to the solutions of (54) may now be obtained 
by substituting unbiased estimates for the unknown values of the ш’ іп (52) 
and (53), substituting the resulting numerical quantities back into (54), and 
solving the resulting simultaneous linear equations for Co, C; , С, and С. . 
The exact extent to which terms of higher order in n can be neglected in this 
process is hard to state for any given value of n. There are good indications, 
however, that terms of order пг! should be retained in (52) and (53) even 
when т is as large as 100. 
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A STATISTICAL TEST FOR THE SIGNIFICANCE OF А 
COEFFICIENT OF REPRODUCIBILITY 


Рнилр С. SAGI 
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Under the assumption that responses to different dichotomous items 
are statistically independent, exact distributions for coefficients of repro- 
ducibility, СВ, are derived. These distributions are useful in testing whether 
an observed coefficient of reproducibility differs significantly from chance 
expectations and whether further scaling manipulations are warranted. The 
effects of purification on sampling distributions of errors are calculated to 
demonstrate а relationship between scaling operations and expected chance 
values of the CR. 


The practice of devising а measuring instrument on the very sample 
that is being used to test à substantive hypothesis involving the measure 
is suspect. The permissive nature of sealogram analysis, the collapsing of 
item categories, the elimination of items if they contribute too much error, 
and the ad hoc determination of positive and negative response categories 
capitalize on the unique and chance features of the sample. The more such 
operations are performed in the process of scaling the more suspect is the 
resultant claim of a scale. At some point evidence should be presented which 
demonstrates (i) that scaling operations were warranted, or (ii) that the 
parallelogram formed by the response patterns to the retained set of ordered 
items is not merely an artifact of the operations performed in the analysis 
and the peculiarities of the sample—but that the items do in fact form a 
scale with the claimed theoretical properties. Such evidence may be gained in 
either of two ways—by replication on à subsequent sample or by the use of 
appropriate tests of statistical significance. Test by replication is neither 
convenient nor possible at all times. ў Wr 

One weakness of current sealing practices in sociological and psychological 
research is due to the lack of knowledge about sampling distributions of 
evaluative summary statistics, such as Guttman's Coefficient of Reproduci- 
bility, CR. The intent here is to present exact distributions and exact tests 
for the CR under certain conditions and to discuss tests of significance for 
other evaluative statistics. This is not an original problem. Guttman provides 
an estimate of the upper bound to the standard error of the CR, given certain 
assumptions ([3], р. 77). This estimate 18 based upon an intuitive appeal to 
the “sampling distribution of ordinary proportions (which follow the binomial 
distribution) especially when the population reproducibility is high (say, 


over .90)” ([4], р. 279). Loevinger recognizes the existence of the problem 
. ‚ P- 
19 
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([6], p. 37). The current practice of reporting the chance value of the CR, 
under the assumption of complete independence, implicitly assumes the 
existence of а sampling distribution of the CR. Further evidence of the 
type mentioned can be readily cited. 


Reproducibility, Homogencity, and the Concept of Error 


Loevinger and Guttman have approached the problem of measurement 
in a manner that emphasizes the patterns of responses to 
items. In the case of dichotomous items with common cont 
by face validity, items are ordered by their difficulty. The ranking by difficulty 
is equivalent to the ranking of items by frequencies of positive endorsements. 
The pattern of responses to ordered items is then compared to an ideal 
pattern of responses. Loevinger's Coefficient, of Homogeneity and Guttman's 
CR are summary indices designed to reflect the totality of divergences from 
ideal patterns. If a response pattern for items ordered from “hard” to "easy" 
+—+-+——+, Loevinger would assign the number 7 às а measure of 
divergence and Guttman would assign the number 3. The numbers 7 and 3 
describe the number of errors that would be associated with the observed 
pattern by the respective systems of measuring divergences. It is clear that 
the definition of error is somewhat arbitrary and is imposed on the data. 


Working rules for counting errors, when dichotomous items are ordered from 
hard to easy, are, respectively: 


a given set of 
ent, as judged 


I. Loevinger—the number of (+ —) patterns among all pairs of items; 

II. Guttman—the minimum number of substitutions of + for — or 

— for + in order to transform the observed pattern into some ideal 
pattern or scale type. 

The two rules have a great deal in 
of (+ —) patterns. The rule for Guttman scales may be viewed as the count- 
ing of (+ —) patterns for adjacent items plus double and triple to n-order 
error patterns among adjacent items, 


The (+ + — —) pattern i ble 
error of the (+ —) pattern, and the (+ + + -- 5. ‘a 


common. Each involves the counting 


- к а triple 
error of this (+ —) pattern. The illustrative pattern +- фф а 
recognized as composed of а Single and a double error pattern, which total 
to three errors. ? 


Additional arbitrary rules for counting errors сап be readily introduced. 
Аз an example, and because it appears fruitful, consider а rule III and count 
as errors the (+ —) patterns among adjacent pairs of ordered items, (Green 
[2] anticipated rule III with minor variations. See [8] for additional rules.) 
Thus, + — + + — — + would be assigned the number 2 as a measure of 
divergence. The observation that this arbitrary rule for counting errors gives 
the same results as does the Guttman system of counting errors when the 
number of dichotomous items is three or less and agrees closely when the 
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number of dichotomous items is four, suggests the utility of some form of 
four-fold analysis involving each pair of adjacent items. 


Exact Distribution of Errors under Rule III 

The expression р' = (ri 1ғ 1с, 1621)/ (alb!c!d!n!) is well known and is the 
probability of the observed four-fold distribution under the hypothesis 
of statistical independence. Row and column totals are treated as fixed and 
not subject to sampling variation. This is also a problem in sampling without 
replacement ([5], pp. 230-231). 

Given K items ordered by difficulty from hard to easy, the probability 
of the observed response patterns, assuming statistieal independence, for 
each adjacent pair of items is 


This follows directly from the assumption of independence, where i denotes 
the four-fold table formed by the ith and 4 + 1 item ordered by difficulty 
from iz = 1,2, °°: , К. Table 1 is an example of the typical four-fold table 
formed by adjacent items in which с; is the frequency of (+ —) patterns 
and is therefore the frequency of errors. (In the case of a perfect scale, the 


TABLE 1 


Typical Four-fold Formed by Adjacent Items 


Item 


іжі Item i response 
response categories 
categories ў x 
Ы ы! bi тү) 
s E E: ШИ] 
өзі < n 


zero frequency and therefore УЕ 0, = 0.) In fact 


K-1 
Пр: 


i=l 


c; cell contains & 


1 ability of exactly бі, Фо 077) cy, errors occurring under the 
LA scene t andit bd Referring to Table 1 while employing 
Fisher’s exact test, the probability of obtaining а frequency in the c; cell less 
than or equal to с; is given by the expression p; — Phi pod Pii +-+ 
та, and the probability of the total errors being less than or equal to Ё сап 


be expressed as 


$ 377 
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K-1 
(1) Prob (с, + €; + ej + +++ + ск, < В < У (П pts), 
where >_., symbolizes the summation for all possible c; such that 
K-1 


2d X E 


181 
Exact Distribution of Guttman’s Coefficient of Reproducibility 


Restricting the argument to four dichotomous items, the probability of 
exactly с, , с» , сз, С, observed errors among four ordered items when errors 
are counted in keeping with the rule covering Guttman scales is 


3 b 
Пи. ("4 ыш - 25 йа — Г | 
эн | TERME d І AU = в —2 


(2) im] І, = (\ | 
tal 
Co Ns — Cz 


where b is equal to the smallest of the three frequencies n, , c, , Or M, — с 
(since G < ns, G < с, and G < m, — cı); m, is the frequency of positive 


identical to the product factor in (1). The remaining factors of 
viewed as a correction for double errors. Formula (2) is readily apparent if 
for each particular array of errors c, , c, ; €; , there exists а sampling dis- 


tribution of double error patterns distributed in the same manner as (+ —) 
patterns for three adjacent items. 


(2) may be 


III for the measurement of divergences. 

In Tables 2 and 3 exact distributions 
according to Guttman and rule III. Table 2 compares exact distributions for 
the two systems of counting errors given a sample of size 10, four items and 
Mı , Mz , Тз , Та, equal to 2, 4, 6, 8, respectively. Table 3 калий — s 
tributions for the two systems of counting errors given a sample size № = 10 
four items (К = 4), and all item marginals equal to five 
marginals are all equal, items cannot be ordered by marginals. However 
we can assume either an a priori ordering or marginals that diff er by 
small numbers. 'These parameter values were chosen for illustratiy 
and ease of computation. 

The columns headed cumulative percentages provide the information 
for one-tailed tests of significance for the evaluative statistic CR. Specifically, 
the cumulative percentages indicate the chances of getting a CR as large or 


are presented for errors counted 
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larger than the one observed if the hypothesis that responses to items are 
statistically independent is true. Rejection of the null hypothesis in favor 
of the alternative hypothesis, the hypothesis that the CR is greater than 
values usually observed if the null hypothesis is true, is taken as evidence 
that the item response patterns are in the direction of a scale. Thus, assuming 
statistical independence and given the parameter values аз indicated in 
Table 2, values of CR greater than or equal to .90 occur with probability 


less than or equal to .6570 or .6354, depending on the system of counting 
errors. By definition 


EO 
^ NK 

In Table 3, the probabilities corresponding to а CR of .90 are .016606 
and .007069. The differences between the probabilities indicate the similarity 
of results between the two Systems of counting errors (at least for four-item 
scales) and the important effect of the item marginals on the distributions. 
In Table 4 the values of E counted in accord with rule III for given probability 
levels under the hypothesis of Statistical independence are tabulated. The 
values of N, K, and m; respectively, are 100, 4, and 20, 40, 60, 80. С 
between Tables 2 and 4 provide the additional and expected буя 
the size of the sample is also an important consideration, 

While the shape of the sampling distribution varies with N, K, and 
m; @ = 1, 2, --- , K), the distributions approximate normality when N 
becomes large. Goodman [1] has demonstrated this latter point and has 
shown the adequacy of the а small sample problem. 


келді ae Es warranted when the m;/N 
азе, radi "tur approxi- 
mate normality obtain with extreme item harha а i 
known over a range of values for N ; K, and т, , may delimit the Bitumtions 
in which Goodman's approximate method yields accurate probabilit естй 
ments. Тһе reader is referred to [1] for à more adequate tr К "4 f the 
large sample problem. ИЯ 


СВ = 1 


omparisons 
mation that 


Discussion 

Under the conditions described, exact tests are generall licable if 
errors are counted according to rule III, For the Guttman 2 5 ical id 
stricted to 2, 3, and 4-item scales. There are practical шы ы: d boue 
be considered. When М is large, greater than 10, and К is 4 7 ni , the 
computational labor involved in the use of the derived жы 5 d 
excessive. The utility of the formulas is further limited by the | as res oe 
ment that the correct application of the tests occur prior to “ ogical a 5 
the practice of eliminating items which purification, 


P ch contribute the greatest numbers of 
errors. The common practice in sealing is to purify and then report the 
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TABLE 4 


Exact Distribution of Errors 
(55100, m,*20, my*40, m3=60, m,=80) 


Number of Relative Cumulative СК 
errors frequencies percentages 
24 + 0000006 .000+ +9400 
25 .0000277 003 .9378 
28 .0000777 өш .9350 
27 .0002203 .033 .9325 
28 .0005522 .088 .9300 
29 .0012905 EX .9215 
30 .0027770 «495 .9250 
3i .0055665 1.051 .9225 
32 .0103488 2.086 .9200 
33 10179269 3.879 19115 
34 .0288232 6.161 .9150 


aaa 


d for the purified scale. Obviously, a test of the 
purified-scale statistic must take into consideration the number and marginals 
of all items prior {о elimination. Formulas (1) and (2) may be used to test 
whether a set of items is worth purifying and scaling. The importance of the 
argument cannot be understated. Even if the hypothesis of statistical in- 
dependence is true for responses to some original K’ items, it is likely that 
purification will yield а subset of К items (К < К’) that appear to scale. 


evaluative statistic calculate 


TABLE 5 


Exact Distributions of E Following Purification 
(N = 10, K = 4,m,=5) 


n 


шы Guttmants system Rule III E 
errors Hypothetical Cum. Hypothetical Сит. 
E frequencies frequencies 
o 1 .000 1 .000 1.000 
1 50 .003 15 .004 .975 
2 1,100 .058 2,115 .M3 .950 
3 13,625 4139 30,625 1.643 4900 
n 90,000 5.243 217,500 12.516 .815 
5 304,000 20.440 150,000 50.009 .875 
6 811,500 64.007 1,000,000 100.000 .850 
7 120,000 100.000 .825 
Total 2,000,376 2,000,376 
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Had formulas (1) and (2) been applied prior to purification, the tests of 
significance would have indicated, with predetermined probabilities of error, 
that purification was or was not justified. Illustrative of this point is the 
effect that purification has upon the expected chance value of the CR as 
well as the sampling distribution of E. 

Assuming statistically independent responses to different ordered dicho- 
tomous items, the expected frequencies, c; , for any pair of adjacent items 
may be computed by the formula m;n;,;/N. If, among other equally arbitrary 
standards, the criterion of purification employed is the elimination of an 
item or items yielding observed frequencies c, > т.т: /N, the remaining 
items must exhibit frequencies c; < m;n,,,/N. Further, if it is assumed that 
the original pool of items is sufficiently large, it is inevitable that some subset 
of K items will satisfy the purification criterion. Exact distributions for 
such purified scales are presented in Table 5. Distributions were caleulated 
Írom (1) and (2) subject to the restriction, c, « таша/М. Purification 


tends to increase the magnitude of the expected CR under the hypothesis 
of statistical independence. 


The expected CR's following purification 
to the unpurified scale statistics of .812 an 
chance values of CR that are commonly rep 
operations performed on data and actually 
chance value of the CR. Moreover, 
purified sets of items increase the ch 
Had some other criterion of purifica 
of errors would have followed. 

Other limitations on the use of these formulas also involve operations 
commonly associated with the scaling process. The stipulation that positive 
and negative response categories be determined by face validity prior to the 
inspection of inter-item relationships is not unlike the restriction that formulas 
(1) and (2) are appropriately used prior to purification. The ad hoe procedure 
of determining sign by examining the direction of association with other items 
under consideration also tends to result in observed с; being less than or equal 

to expected с, frequencies. Similarly, the practice of collapsing item-response 
categories to form contrived dichotomous items is not an admissible operation 
if such collapsing is done on the basis of observed inter-item response patterns.* 

The conditions under which tests of significance apply define prior 
admissible scaling operations. Conversely, different Sealing operations usually 
imply different sampling distributions of the evaluative Statistic. The impli- 
cation is immediately apparent: cither scaling operations are restricted to 
those operations consistent with developed statistical tests, or extensive 
replication is used as a substitute testing procedure. Tests proposed in this 


*Interestingly enough, such ad hoc collapsing exploits any lack of ап isotropic 
relationship between adjacent items, a situation that is inimical to the concept of a scale. 


are .866 and .848 as compared 
d .797, respectively. Expected 
orted do not correspond to the 
underestimate the true expected 
applications of formulas (1) and (2) to 
ances of falsely rejecting null hypotheses. 
tion been invoked, a different distribution 
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paper and in [1] merely serve to establish whether responses to a set of ordered 
dichotomous items are worthy of further manipulation. The complete rational- 
ization of the scaling process requires a test designed to differentiate between 
artifact and true improvement of the CR following scaling manipulations. 
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SIMPLE STATISTICAL METHODS FOR SCALOGRAM ANALYSIS* 


Іво А. Соормамї 
UNIVERSITY OF CHICAGO 


Simple statistical methods are developed to test whether coefficients of 
reproducibility, of homogeneity, or of consistency differ significantly from 
what can be expected if responses to different items are statistically inde- 
pendent. Simple methods are also developed for estimating the variance of 
coefficients of reproducibility when it is not assumed that responses to dif- 
ferent items are independent. These estimates are used to test whether a co- 
efficient differs significantly from any prescribed value, and also to obtain 
confidence intervals for these coefficients. The rationale for the measure- 


ment of reproducibility is also discussed. 


The desirability and importance of developing statistical methods in 
scalogram analysis for the comparison of an observed coefficient (of re- 
producibility) with its expected value, estimated under the assumption that 
responses to different items are statistically independent, has been pointed 
out in [4] and [7]. In the following three sections such statistical methods are 
developed for the following coefficients: (а) Loevinger's coefficient of homo- 
geneity [10], (b) Green's coefficient of consistency [6], (c) а coefficient. of 
reproducibility given by Green [6], and (d) а coefficient of. reproducibility 
given by Sagi [13]. These methods can be used to test whether each of these 
coefficients is significantly different from what can be expected if responses 
to different items are statistically independent. Of course, these methods are 
to be applied to a given coefficient only in situations where it seems appropri- 
ate to use that coefficient [cf. 6, 10, 11, 13]. These methods can be used to 
determine "whether a set of items is worth purifying and scaling" [ef. 13], 
and should not be applied, except with great caution, to а set of items that 
have been purified. 


Although the comparison of an observed coefficient with its expected 


value, estimated under the assumption that responses to different items are 
independent, is useful for some purposes, this particular kind of comparison 
may be neither very informative nor useful for other purposes. It may be 
more or less obvious that the responses are not independent and that а 


i i isti arch Center, University of 
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бысаш under sponsorship of the Statistics Branch, Office of Naval Research, and of the 
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birthday, July 23, 1958. 
29 


30 PSYCHOMETRIKA 


coefficient does differ significantly from what can be expected if independence 
were assumed. It may be more worth while to compare the observed data 
with some model other than that obtained by the assumption of independence. 
The problem of testing whether a coefficient differs significantly from what 
can be expected under the assumption of independence may not be identical 
with the problem of testing whether the observed data indicate that there is, 
in some sense, a scale. Also, the problem of measuring the size of the difference 
between an observed coefficient and what can be expected in the case of 


are discussed briefly, and methods are presented for esti 


mating the variance 
of Guttman’s coefficient, or of a number of other с 


oefficients, of reproduci- 
nce is not made. These 


of situations discussed in 


[6, 10, 13]. For the sake of completeness, the situation in which there is à 


random sample of items.also is considered. 
Finally, a numerical com 
suggested in the fol owing secti 


greater than 10, and K (the number of item 


number of respondents in the sample (i.e., th 
be used without any real difficulty, for any nu 


The methods presented in the 
sense that they depend on certain 


suggested. It would be valuable to compare the various appro 
presented here in many other particular cases in order to 
large а sample must be drawn for these methods to lead to 
mations; this would be worth while with regard to the appro 
presented here as well as to many other statistical methods 


useful approxi- 
ximate methods 
(ef. [2], р. 328). 
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A Coefficient of Reproducibility 

РА E аа лы V erg i all defined for an ordered set of 
articles it was assumed that is үзсө ав Нана pes de wm 
a priori knowledge [cf. 11, 13], or b as E = epis ча pos 
2 1 g - 11, 13], e determined directly by arranging ‘ће 
items in rank order according to their popularities" [6]. It also will be assumed 
here either that the ordering of the items is known a priori, or that the total 
number of respondents sampled is sufficiently large so that almost certainly 
the rank order according to popularities will be the same as that obtained 
if the total population of respondents were studied. In the latter case, it is 
assumed that the rank ordering according to popularities obtained for the 
total population is unambiguous (i.e., that the “popularities” of two different 
items will not be exactly equal for the total population) and that this rank 
ordering corresponds to the “true” ordering of the K items. The situation 
where there are more than two kinds of responses to each item was com- 
mented upon in [6, 7, and 9], and it will not be discussed here. 

For K = 5, the response pattern (+—++-—) indicates that the re- 
spondent gives positive responses to items 1, 3, 4, and a negative response to 
items 2, 5. The following response patterns would be ideal, indicating а 
"perfect scale" [ef. 4: (----- ) (----+), (---++), 
(==+++), (=++++), (+++++). An observed response pattern 
may differ from the ideal patterns; there are several ways of measuring this 
“error.” 

In [13], the total error e’ is obtained by comparing item $ with item 
i + 1, counting the number of (+—) patterns observed for these adjacent 
items (1.е., the number fi, +1 of individuals who answered positively on item 
i and negatively on item $ + 1), and summing these counts for all 2; i.e., 


к-1 
е! = 5 ШЕТ 


ізі 

g the total error is identical with Gutt- 
cient of reproducibility is based directly 
1 — (e/NK), а method for testing 


For К < 3, this method of computin 
man’s method [ef. 13]. Since the coeffi 


on the observed total error, i.e., с’ = 
whether the observed e’ differs significantly from what can be expected when 
the responses to different items are independent will be given first. If the 
number of individuals who answered item 2 positively is f; , and the number 

then the expected value 


of negative responses on item 2 EN — fe = Fea, қ 
ee. f; and fi) of hon under the assumption of independence, is f:fi+ı /N. 


Also if the assumption of independence is true, using the usual approach to 
the problem of testing the null hypothesis of independence in a 2 x 2 con- 
tingency table (or the null hypothesis that the expected value of the difference 
between two proportions, obtained from independent samples, is zero), it 
can be seen (еі. 12] that the distribution of the statistic [fiii — (А N)T/ 
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d dia: 3) will be approximately a x^ distribution with one degree of 
Loma ge ота infinity. Furthermore, the distribution of the 
statistic [fi + — О ИМ) Јаја) will be approximately & 
normal distribution with zero mean and unit variance, if the assumption of 
independence is true, and № — < (cf. [12], p. 71). It can also be scen that, 
if the assumption of independence is true, the distribution of the statistic 


р> [fca = Gefen / N] 
(= ҮТЕ ЖЫ 


will be approximately a normal distribution with zero mean and unit variance 
(see Appendix). Thus, the null hypothesis of independence of response ean 
be tested by seeing whether e' differs significantly from 


K-1 

B = M ha; 
4-і 

ie., by computing (е — E^)/8,. = #', where 
1 


SG = 2 Би а №, 


and comparing the numerical value of z' with the standard unit normal dis- 


tribution. For example, an approximate one-sided test, at the 95 per cent 
level of significance, can be obtained by rejecting the null hypothesis when- 
ever 2’ < — 1.645. 


Since c’ = 1 — (e'/NK), z = —(e — C’)/S.. , where C = | — (КҮК) 
and Se = S,./NK. Thus, an approximate one-sided test, at the 95 per cent 
level of significance, can be obtained if the null hypothesis is rejected whenever 
c > C + (1.645)5,.. 


A Coefficient of Homogeneity 
Rather than compute е’ as was done above, с 
error €" obtained by comparing item 7 with item j 
number of (4- —) patterns observed for these tw 
f;.; of individuals who responded positively on ite 
j), and summing these counts for all 7 


onsider now the total 
(4 < 4), counting the 
0 items (i.e., the number 


m 7 and negatively on item 
and j (i < 7; ie, 
K-1 K 
e" = x > fap 
ізі j=i+1 
The expected value (given the f;) of e”, when responses to different items 
are independent, is 


E” = 2, >; EIN. 


ізі ігізі 
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Loevinger suggests (Е” — e’)/E" as a coefficient h of homogeneity (еі. 10, 
15]. A simple test of the null hypothesis that responses to different items are 
independent is based on determining whether h differs significantly from zero. 

By an approach similar to that presented above (cf. Appendix), it is 
seen that, if the null hypothesis is true, then the distribution of the statistic 
(Е”- e")/8,.. = 2" will be approximately a normal distribution with zero 
mean and unit variance, when № — о, and 

Roi OK 


S. = X X QN). 
i=] ізізі 
Thus, an approximate one-sided test, at level of significance .95, can be 
obtained by rejecting the null hypothesis whenever z” > 1.645. Also, since 
h is such that hE”’/S,. = =”, an approximate test of the null hypotheses at 


level of significance .95 can be obtained if this hypothesis is rejected whenever 
№ > 1.645 S../E”. (For this hypothesis, the expected value of h is zero.) 

The method presented here is an approximation in the same sense as is 
the method presented in the earlier section. The results of some numerical 
comparisons presented in the final section tend to suggest that the approxi- 
mate test of whether h differs significantly from zero will be quite useful. 
The modification described in the final section to improve the approximation 
when sample sizes are small or moderate can also be applied to obtain from 


S. a modified statistic 55). 
Another Coefficient of Reproducibility 


Now consider one of the coefficients Rep; in [6], suggested as an estimate 
of reproducibility. (The author states that this coefficient was checked 
empirically, and that the average discrepancy between this coefficient and 
the “correct” coefficient of reproducibility was .003 (еі. 61.) For this coefficient, 


the total error e” is obtained as 
K-1 K-2 
е!!! = >; fiizi + > (аланы ИМ, 


The expected value (given the f;) of е!!! under the 


and с" = 1 — (е'''/ ҮК). \ ‹ 
different items are independent, is 


assumption that responses to 


K-1 cT = Ез 
> fifin >; Раја 


ЕП = Ин ү 


l| hypothesis that responses are independent is to 


determine whether е”! differs significantly from E". 
hat presented above, it can be seen that, if 


By àn approach similar to t ENS SENSE у 1 
the hypothesis of independence is true, then the distribution of the statistic 
(E — 6") 8,0, = z will be approximately normal in form with zero 


A simple test of the nu 
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mean and unit variance when № — о, and 


K-3 
ӨЗ,» = SS. + (5 IS е ндай a Бон | 


і-2 


+ St ба + iaa e UN 
where 
EQ; = FEIN and Si; = HIN. 


(This result is derived in the Appendix.) Thus, an approximate one-sided 
test, at level of significance .95, can be obtained by rejecting the null hypoth- 
esis whenever z"' > 1.645. Also, since c" = 1 — (e"/NK), 2!" = (c — 
C"/)/8,... , where C" = 1 — (E"/NK), and Ser = 8,../МК. Hence, an 
approximate one-sided test, at the level of significance .95, can be obtained 
if the null hypothesis is rejected whenever c" > C" + (1.645)5..... 

Green [6] also suggests an index of consistency g = (с — Qi — 6"). 
Since g(1 — C'7)/8.... = а”, an approximate test of the hypothesis of 
independence at level of significance .95 сап be obtained if this hypothesis is 


rejected whenever g > 1.645 5..../(1 — С”). (For this hypothesis, the 
expected value of g is zero.) 


Some Different Problems and M. ethods 


In the preceding sections, all of the formulas for the expected values 
(the E's) and the 8% were computed under the assumption that responses 
to different items are statistically independent, and the large sample dis- 
tribution theory results also were obtained under this assumption. Now this 
assumption will not be made, and a somewhat different problem will be 
studied. The methods presented in this section can be applied to Guttman’s 
coefficient of reproducibility, as well as to а number of other coefficients, 
and to the problem of testing whether the expected value of a given co- 
efficient is some prescribed numerical value. Als 


| т , о, these methods сап be used 
to obtain approximate confidence intervals for th 


я е expected value of the 
coefficient. 

Consider the situation where there is an ordered set of items, with only 
two kinds of responses to each item. Assume that а, sample of теврондеп&8 
was drawn at random from an infinite population о 


| r 1 т, for a finite population, 
that sampling was done with replacement. Since there are 25 different possible 


patterns of responses, the total population could Conceivably be divided up 
into 2* different categories corresponding to these patterns. To each category 
i.e., to each pattern of responses, a number is assigned indicating how different 
that pattern is from an ideal one; e.g., to pattern (++— — —) the number 
1 is assigned for the method in [13], and the number 2 is assigned for Guttman’s 


— "— 
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method [ef. 13]. The latter method is referred to as Guttman’s method in 
[13]—this title is also used here although it may differ somewhat from the 
methods in [7] and [8]. The different methods lead to different coefficients of 
reproducibility. The number assigned to an ideal pattern, e.g., (——— ++), 
is zero. Hence, for any given method of assigning numbers, the (th pattern, 


і = 1,2,5, 2*, will be assigned a number X, , and the population co- 
efficient of reproducibility is 


25 D, X, 


pe NK 


where D,/N = P, is the proportion of the population in category t. The 
coefficient in [13] and also the coefficient obtained using Guttman's method 
are both of this general form; Green's coefficient Reps, i.e., с””, differs some- 
what from this form, but his coefficient Rep, is also of this general form 
[ef. 6, 13]. All of the coefficients mentioned here of this general form have 
the property X, < K/2 for all ¢; e.g., the maximum value of X, , for these 
coefficients, occurs when the pattern is of the kind (4-— 4- — + —), where 
X, = K/2 (for К even). Thus, Y, = Х./К < 1/2, and 


Ур, У, 


= 1 -+ -1- Pues 
9 А N УР, 


The form of the coefficient W, presented by Jardine in [9], is quite 
different from the general form described here. The coefficient W measures a 
different aspect of the data, although this coefficient can be used to develop 
still another test of the null hypothesis discussed in the preceding sections, 
i.e., that responses to the different items are statistically independent [cf. 
9]. Jardine mentions that “a test for W іп the non-null case is not available.” 
In the present section, tests in the “non-null case” for coefficients of the 
general form described above are presented. 

If С is understood to be a measure of scalability (or a definition of the 
amount of scalability), then it is clear that the system of numbers X. (or 
the Y.) is а basic part of this definition. For different systems of Y, , different 
definitions of scalability are obtained. Which system (ie., which measure 
or definition) will be appropriate depends on the particular context to which 
the measure is to be applied (cf. discussion of measures of association in бі), 
For example, in some context C*, based on using Y, — 1/2 {ог all categories 
t that do not correspond to ideal patterns, will be appropriate. This measure 
is closely related to the proportion of the population falling into all categories 
that correspond to ideal patterns, and C* will be useful if the context is such 
that the important factor is this proportion. If prediction of a response 
pattern for an individual is to be based on his total score, and on the ideal 
pattern corresponding to that total score for the set of items, then the 
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proportion of correct predictions (of response patterns) for the population 
will be 2C* — 1. Thus, for some purposes 2C* — 1 (or C*) measures an aspect 
of the reproducibility of the response patterns from the total scores and the 
set of ideal patterns. 

In some contexts, it may be preferable not to use the collection of K +1 
ideal patterns as the possible predicted patterns, but rather some other 
collection of patterns as the predicted ones. Thus, sometimes the proportion 
of correct predictions of response patterns for the population can be increased 
by using the K + 1 patterns that occur most frequently in the popul 
the collection of possible predicted patterns. Also, for some purposes, this 
collection need not consist of K + 1 patterns; perhaps some larger number 
would be worth while. Reproducibility in the sense described above сап 
sometimes be increased if the number of possible predicted patterns is in- 
creased. Hence, reproducibility in this sense can be high in situations where 
C* or some other measure of scalability is low. It mig 
to use the term coefficient of scalability 
given in [6, 7, and 13]. However, 
operational concept that is worthy 
proportion of item responses, 
predicted correctly, will be i 
an appropriate measure, but 

An estimate of the coe 
random sample of N respon 


ation as 


ht have been preferable 
for the coefficients of reproducibility 
the term reproducibility does suggest an 
of emphasis (еі, 5]. In some contexts, the 
rather than the proportion of response patterns 
mportant. For these situations (* would not be 
some other coefficients might be (ег, 7, 8, 15]. 
fficient of reproducibility in the 
dents is observed, would be 


Уа, х, Pd, Y. 
ME о ані. 
where d,/N is the proportion of the & 
of c is C, and the variance of c is 


| LP. Yi (УР, у 
с. = a —-. 


case where а 


с = 1 


ample in category l. The expected value 


Since Y, < 1/2, 
: УР, Y,/2 == Oe P; о” (1 = C)/2 - 
< П = 


б. 


(1 — (9° 
N 7 


=> N 


_ ОбС 2 

А, 
This gives an upper bound 27 for the exact variance c? of the coefficient of 
reproducibility. Гог some coefficients (1.е., for some assignments of numbers 
Y), the exact variance в will be very close to the upper bound 2° (e.g., for 
c*, o? will be equal to б°). For other coefficients, the bound 27 may be much 
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larger than c? . For any coefficient, c? will be equal to 2^ if P, = 0 for all cate- 
gories ¿ where the assigned numbers У, differ from 0 or 1/2. When c? > 0, 
using the usual central limit theorem, the distribution of z = (c — C)/c. will 
be approximately normal with zero mean and unit variance, when № — œ. 
Also 2 = (c — C)/é., where 


Уа, Y? - (52d. ИМ 


T П 


i" NW = 0 , 


will be approximately normal with zero mean and unit variance. Further- 
more, 2 = (c — C)/é and 2 = (c — C)/8 , where # = (1— о) (c — 1/2)/N, 
will be approximately normal with zero mean and variance X 1. Thus, if 
the null hypothesis is that C is a specified value, then an approximate one- 
sided test, at level of significance .95, can be obtained by rejecting the null 
hypothesis whenever с > С + (1.645) 0. . A simpler approximate (and 
conservative) one-sided test, at level of significance < .95, can be obtained 
by rejecting the null hypothesis whenever ¢ > C + (1.645)5. For some 
purposes, а two-sided test may be appropriate; the general method described 
herein also leads directly to these two-sided tests. Using this method, it is 
also possible to obtain one or two-sided confidence intervals for С. Thus the 
interval e — (1.96) 4. < C < € + (1.96) в. is an approximate 95 per cent 
confidence interval for C. Also, the interval c — (1.96) & < С < c + (1.96) в 
is an approximate (conservative) confidence interval at a level € .95. 

The preceding comments pertain to the situation where a random sample 
of individuals is drawn, and the numerical value of X, can be determined for 
tth response pattern, ¢ = 1, 2, 77 » 2*, For a fixed, i.e., given, response 
pattern 1, the number X, is determined by the definition of scalability used, 
and this number is considered fixed. In some situations, it may be worth while 
to consider this number as а random variable z, rather than as a fixed value 
X, . This will be the case if X, is the number assigned to the tth category 
defined for a given population of K items, and тұ і Ше number obtained for 
this category when a random sample of Ё items is drawn from this population 
of К items (k < К), and the response patterns to these sample items are 
used to compute x, . The case where there is à random sample of items will be 
discussed for the sake of completeness, although in many situations it may 
not be appropriate to consider the actual set of items used as a random sample 


from a larger population. 


In this case, if the coefficient is computed on the basis of the observed 


response patterns to the k sample items, then this coefficient will in general 
differ from the corresponding coefficient, for the population of K items. In 
the special situation where the K items form a perfect scale, the coefficients 
would be identical. In fact, the expected value of the coefficient based on k 
items, for all possible samples of k items, will generally differ from the co- 
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efficient based on K items. For example, in the extreme сазе where a popu- 
lation consists of K — 4 items, and where (+—+-—) is the only response 
pattern, then the population value of 2C* — | — 0, but this measure computed 
from a random sample of two items drawn from the 4 will take on the values 0 
or 1, depending on which two items are drawn. The expected value of this 
measure will be .5 when items are sampled without replacement. In this 
extreme case, C', C*, and Guttman's coefficient will all take оп the values .5 or 
1 depending on which two items are drawn; the expected value of these coeffi- 
cients will be .75, while the population coefficient is .5. Thus, the coefficients 
of reproducibility computed for a sample of items from a | 
items must be interpreted with great caution. They gener 
unbiased estimates of the correspondin 


are used in an empirical 
S а population of M sets, 
sets of items is randomly 
the situation where the 
f items are obtained, and 
ion of sets of items for this 
is the proportion of sets among the M sets where the 
response pattern for individual j corresponds to the tth category, for = 1 
2, --- , 2". Thus, in an i get еле where a single set of K iles is used, 
1 = < — it corresponding to the sth individua: 

population, where ,Y = ;X/K is the number asse амы 
mw PW enia for the jth individual can be considered an estimate 

This c; will be an unbiased estimate of C; and the varianc 


is У, вт ix (5. 52° =й, о en € of this estimate 


M 


Ox = > (c - OY/M, 

m=1 
where с” denotes the coefficient of reproducibility computed for the mth 
set (m = 1, 2, 994. M ) of ordered items in the population of M sets of items 
In the special situation in Which, for each respondent in the арбоо »* 
Д 


set of items is drawn at random from the population 

e of sets i 2 
ability of response pattern ¢ for individual j is R, ‚ and eos vu as 
independent of the response pattern obtained for а different individu 1) md 
the variance of c will be somewhat different; viz., а» 
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cx = У) 05/0 = ау/М, 
where 22 is the average value of ;o} for the total population. 
For the case in which a random sample of № respondents is drawn from 
a large population the coefficient of reproducibility c computed for the sample 
responses to a single set of items will be an unbiased estimate of C; the variance 
of c will be 25, + (c2/N), where ег is the variance of C; for the total popula- 
tion of respondents, 25, is the average of 


м 
Ж, (с) — @.)/М = (oar 
mel 


for all samples (s = 1, 2, ---) of size М, and c? and C, are the corre- 
sponding values of c™ and C computed for the sth sample, by considering 
this sample as а population of size N. Also, in the special case where the 
probability of response pattern t for individual j is ;R, , and this probability 
is statistically independent of the response pattern for а different individual, 
then the variance of c will be (22 + «c)/N. In this case, if it can be assumed 
also that ;, = R, , thus, C; = C and jo2 = о forall individuals j = 1,2, кн» 
then the expected value of the coefficient of reproducibility c, computed for 
а single set of K items for a random sample of № individuals, is C; the variance 
of ciso?/N = c;. The latter was computed earlier in this section; an upper 
bound for c? was also presented. 


Some Numerical Comparisons 


In [13] the exact distribution of e’ is presented for some special cases. 
Now the results obtained using these exact distributions will be compared 
with those obtained using the approximate methods presented in earlier 
sections. 

As seen in Table 4 in [13], if М = 100, f, = 20, f, = 40, fs = 60, f. = 80, 
then the null hypothesis that responses to different items are statistically 
independent would be rejected at the level of significance > 95 Це < 33. 
At the level of significance > .975, rejection of the null hypothesis would 
occur if e/ < 32. At the level of significance > .99, rejection would occur if 
e' < 30. 

The approximate method described in an earlier section indicates that, 
at level of significance > .95, the null hypothesis should be rejected if 
2 < 1.645; i.e., if е < 40 — (1.645) (3.67) = 33.96. Thus, in this case, the 
approximate test and the exact test lead to the same critical region, e/ = 0, 
1,2, +++, 33. At level of significance > .975, rejection would occur if e' < 40 — 
(1.960) (3.67) — 32.81, i.e., if & < 32; at level of significance > .99, rejection 
would occur if e’ < 40 — (2.326) (3.67) — 31.46, i.e., Не’ X 31. 

From the preceding numerical computations, it is noted that the only | 
difference between the exact method and the simpler approximate method 
is obtained at the level of significance > .99. Actually, this difference at the 
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> .99 level is due in part to the fact that the exact level of significance of 
the critical region e’ < 30 is 1 — .00495 = 995, while the exact level of 
significance of the critical region e’ < 31 is 1 — .011051 = .989 (cf. 13]. Thus, 
the approximate method leads to quite an accurate approximation in the 
particular case considered even at level of significance > .99. In this case, 
the method is conservative in the sense that it will lead to rejection of the 
null hypothesis whenever the exact method would have led to rejection, but 
it may also lead to rejection in some cases where the exact method would not 
(cf. [14], р. 105). 

From Table 3 in [13], ИМ = 10, f; = 5 fori = 1, 2, 3, 4, then the null 
hypothesis would be rejected, at level of significance > .95, if с’ X 4. At 
level of significance > .99, rejection would occur if e < 3. The approximate 
method leads to rejection, at level of significance > 95, if e' < 7.5 — (1.645) 
(1.37) = 5.24, i.e. е < 5; also, at level of significance > .99, the approximate 
method leads to rejection if e $ 7.5 - (2.326) (1.37) = 4.31, ie, е < 4. 
Thus, in this particular case where N = 10, the critical regions obtained by 
this method differed by one unit from the critical regions obtained using the 
exact distribution. Here, too, differences between the results obtained by the 
approximate and exact methods are due, in part, to the discontinuous 
nature of the exact distribution (cf. [2], p. 331). In this case, too, the approxi- 
mate method leads to a conservative test. 

For medium and rather small samples, a modification of the approximate 
methods (changing an N toan М — 1) may lead to improved approximations. 
This modification is based on the fact that, when the null hypothesis is true, 
the statistic 


fian X MFin /N) T 

аЛ — DIP 

will be approximately a unit normal variate 
will also be approximately 


| (cf. [12], р. 71), and (е — B/S 
a unit normal, where i 


Sh = b аў ЛМ = о |“. 


Thus, for the preceding example, 5% = (3(625)/(1000-100)]'/ = 1.44 and 
this modified method leads to rejection of the null hypothesis, at leva of 
significance > .95, Не’ < 7.5 — (1.645) (1.44) = 5.13, i.e., е < 5. Also at 
level of significance > .99, this method leads to rejection if е" < 75 = 
(2.326) (1.44) = 4.15, 16.26 ж 4, Thus, a slight improvement is obtained 
although it is not sufficient to change the critical regions giy 
the S.. rather than S5 . 


Since the approximate methods given here 
same kinds of asymptotic results as the usual 
testing independence in à 2 x 2 


en earlier using 


аге based, in part, on the 
t approach to the problem of 
contingency table, the reader is referred to 
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[2] for diseussion and recommendations concerning the 2 X 2 contingency 
table and also for some references to tables of exact distributions that can 
be of some use in computing the exact distribution of e’ when № and К are 
small. The reader will note that the modification presented in this section 
to improve the approximate methods is not related to the usual Yates correc- 
tion (ef. [2], p. 332). A modification related to the Yates correction is not 
recommended because, for the 2 X 2 contingency table, this correction “Ваза 
t" and this "over-correction mounts up” [2], when 
the corrected statistics for each of a number of 2 X 2 tables are combined. 

It. has been shown that the approximate method for testing the hypoth- 
rent items are independent, and its modified 
form given here, lead to quite useful results in two cases. If N is sufficiently 
large to permit the application of the usual x? test for independence to each 
of the (K — 1) contingency tables obtained by comparing responses for 
itemszand? + 1 (i = 1,2, 77; K — 1), then the approximate method, or 


its modified form, leads to useful results. 


tendency to over-correc 


esis that responses to diffe 


Appendix 


When the responses to different items are statistically independent, 
the statiste (E — еу = 2”, can be shown to be approximately 


normally distributed with zero mean and unit variance when № — о. А 
similar approach сап be used to prove corresponding results for the statistics 


z” and z’. 


The statistic c" — Fi" can be written as 


"= K-2 А д 
e^ — p" = рэ (рат — Bae) F >, (RETO TEE E, лая) / N 
fel =з 


жы K-2 В с 
- T DONT F у (аа а-ға zm Елин Даян) ИМ, 
m $-2 

The expected value (given the numbers fi) of 
ш i ‚ , will be zero, and the 

e" р" wi ero, since the expected value of A;.; WI ero, 
Жж An к Жалынып will be statistically independent (given the f). 

4.42 ыы 


(е" — БМ" = w may also be written аз 


where д;; = fii — Eri 


(-і к-? Ж 
aca: Assisi + »» (Bi ioi + Е, прич 


+ — 


Each A, М- = v; will asymptotically normally distributed with zero 
mean and an asymptotic variance (for fixed number f;/N) of Si, = 1. i, N 
(cf. [12], p. 71; [3], p. 366). Thus, 
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к-і K-2 
w = Db. + 25 (Е, d Каный даа 
i=l i=2 


T Шан ала NN, 


K-1 


Уеа “ыма”. + Фк-з.кЕк-а,к-і 


isl 


K-3 
+ У, лаа + Ei iss) (М + 6), 


where 


K-2 

6 = 2; лаал, 
Since à will converge in probability to zero, the statistic w is essentially a 
linear combination of the 2/5 (cf. [3], p. 254). The v;;’8 are asymptotically 
independent—this follows from the assumption of independence of the 
responses, or it could also be proved using t 


he methods given in [1]. Hence 
the statistic w is essentially a linear combin, 


ation of asymptotically normally 
distributed independent random variables, and this Statistie will also be 


asymptotically normally distributed. The asymptotic varianee (for fixed 
f:/N) of this statistic will be 


к-з 
Sisia аи, + Y. CY TEN рр ЕЁ 


2 
: 441,44 з) 
Т” T м iii pu 
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HIERARCHICAL FACTOR SOLUTIONS WITHOUT ROTATION 


Вовевт J. WHERRY 


OHIO STATE UNIVERSITY 


A method is presented for securing a hierarehieal factor solution which 
achieves simple structure at each hierarchical level without rotation or even 
preliminary arbitrary orthogonal or oblique solutions. The method is based 


upon the assumption that if overlap is removed from clusters the remaining 
specifics will achieve simple structure automatically, The problem presented 
imple structural rotation as 


earlier by Schmid and Leiman, using oblique si 
a basis, is reworked by this new approach. 


Schmid and Leiman [1] recently published an article on the develop- 
ment of hierarchical factor solutions. Their method was an extension of a 
Thomson [2] a number of years ago. It consisted of 
he factors at the first-order level to oblique simple 
structure. The factor correlation matrix was then obtained by inverting the 
reference vector correlation matrix and normalizing it. This factor correlation 
actored and the process was repeated at this higher level. 
This сусе was repeated until a single factor explained the highest level factor 
matrix. A method of back-solution was then provided which yielded a trans- 
formation matrix to be applied to the next lower level oblique simple structure 
loadings for cach order involved. 

The author, with the assistance of Dr. B. J. Winer, had worked out this 
identical extension independently. The method was still considered too 
cumbersome and indirect, however, and the search for a more direct solution 
continued. With suggestions from several sources, particularly from Dr. 
Roderick Bare, the author succeeded in eliminating the need for oblique 
cluster loadings, their transformation to orthogonality, the subsequent 
rotation to simple structure, and the need for the reference and factor corre- 
lation matrices. 

The present method starts with the multiple group factor method, and 
obtains the usual cluster sums and the cluster correlations. It is assumed 
that if all overlap is removed from the clusters then they will have simple 


structure with respect to eac 


method proposed by 
factoring and rotating t 


matrix was in turn f 


h other. Hence the multiple group method is 
applied immediately to the cluster correlation matrix and a new set of higher 
order cluster sums and cluster correlation matrix is obtained. This process 
is repeated until the final cluster matrix consists of a single cluster. 

It is now assumed that this final cluster represents а general factor 
with loadings on all of the original tests or variables, and that the specifics 
at that level represent sub-general factors possessmg simple structure with 
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respect to each other and having loadings on all tests contained in the 
order clusters which compose them. Each cluster, since it represents 
lated reference vector, is assumed to have unit variance for purposes of 
computing the specific loadings; of course, communalities are used in the 
multiple cluster factorization. The data at this final stage will consist of: 

а. the highest level unitary factor cluster matrix, a set of loadings for that 


factor, and a specific factor loading for each lower order cluster represented 
in the matrix; 


b.one or more lower order anal 
available: 


1. the cluster sums for each still lower order cluster (in the lowest order 
these are cluster sums for each original variable), 


the reciprocal of the Square root 
variables in each cluster, which 
elation matrix at that level. 


obtaining а hierarchical Solution identical to 
-Leiman approach. 


lower 
а postu- 


узез for each of which the following are 


were used to obtain the cluster corr 

These data are sufficient for 
that achieved by the Schmid 
The following steps serv. 

(1) Using the highest о 


; and these products are used as a transformation matrix. 
(3) These transformation wei 


of the previous analysis, which yields gene 
for each of the next lower order clusters, 


puted from these factor loadings for each 


ed from unity, and the Square root is taken 
g for the cluster, 


es 1 through 5 


relations (Roo) and communalities 
By Inspection, each Successive pair 
ate cluster GC), indicated by spacing 
d the six sets of cluster sums (,5 ^,0). 


rst-order cluster sums C. 
the reciprocals of the square roots of the major diagonal ns fs s 
С") 
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TABLE 1 


Intercorrelations of 12 Variables with Cormunalities in the 
Diagonals, and Six Sets of Cluster Sums for Each Variable 


Intercorrelations, Roo 
Cluster A B с D E Е 
Variable 1 2 3 å 5 6 i 8 9 10 11 12 
1 6400 7200 3136 2688 0983 0491 1290 0369 2903 1613 0645 0753 
2 7200 3100 3528 3025 1106 0553 1452 0415 3266 1816 0726 0847 
3 3136 3528 4900 4200 0753 0377 0988 0282 2222 1235 0494 0576 
а 2688 3024 4200 3600 0645 0323 0847 0242 1905 1058 0424 0494 
5 0983 1106 0753 0645 6400 3200 1344 0384 1089 0605 0242 0282 
6 0491 0553 0377 0323 3200 1600 0672 0192 0544 0302 0121 0141 
7 1290 1452 0988 0847 1344 0672 4900 1400 1429 0794 0318 0370 
ü 0369 0415 0252 0242 0384 0192 1400 0400 0408 0227 0091 0106 
9 2903 2766 2222 1905 1069 054% 1429 0408 8100 4500 1956 1701 
10 1613 1814 1235 1058 0605 0302 0794 0227 4500 250 0810 0945 
n 0645 0726 04% 0424 0242 0121 0318 0091 1453 0810 3600 4200 
12 0753 07 0576 0494 0252 0141 0370 0106 1701 0945 4200 4900 
Cluster Suns, IE Cie]* 

^ 1.3600 1 5300 6665 5712 2059 1045 2742 0734 6169 3427 1371 1600 
B 502% 6552 9100 2500 1390 0700 1335 0524 4127 2293 0918 1070 
с 1474 1059 1130 0968 9600 4800 2016 0576 1633 0907 0363 0423 
D 1659 1867 1270 1009 1728 0064 6300 1300 1837 1021 0409 0476 
4516 5080 3457 2963 169% 0046 2223 063512600 7000 2268 2646 

EH 303 1573 100 0918 0524 0262 0688 0197 3159 1755 7800 9100 


* Sums for variables in cach cluster are underlined. 


relations (Ва) with communalities in the diagonals. These 


cluster intercorr 
communalities were computed by a modified Spearman equation 


PF 
таа Тук э 


where 7,, and 7;, represent the average correlation of variables in the cluster 
with all variables not in the cluster. Inspection showed this matrix to contain 


three clusters, indieated by spacing in the table. Beneath the intercorrelation 


table are the three rows of second-order cluster sums (У С:С;:). 

Table 3 gives the sums of the second-order cluster sums (2, 2 C:C;-); 
the reciprocals of the square roots of the major diagonal terms (M c;), the 
second-order cluster correlations (№), with communalities computed as 
above. This matrix forms a single cluster, and so this table is followed by a 
row of sums, the actual factor loadings, and the appropriate specific loadings. 
This concludes the first or forward phase of the solution. 

Table 4 shows the details of the second or back-solution phase in some 
detail. It contains a Doolittle solution using the second-order cluster inter- 
correlations from Table 3 with ones in the diagonal, and with the loadings 
from that same table used as successive criteria. The beta weights for each 
criterion, obtained by the usual back-solution method, are then postmultiplied 
by a diagonal matrix of multiplier values (+M), also found in Table 3, to 
obtain a transformation matrix (Tı). Finally the transpose of this trans- 
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formation matrix is used to postmultiply the transpose of the second-order 
cluster sums from Table 2, which yields the factor structure for the second- 
order domain. The general factor is labeled Г, and the three sub-general 
factors are labeled S, , S, , and S.. 

Table 5 presents some of these same steps for the transformation into 
the first-order domain with loadings for the original variables. The first-order 
transformation matrix (Т,), obtained by a Doolittle solution using first- 
order cluster intercorrelations, first-order cluster loadings (including specifies) 
in the second-order domain as criteria, and the diagonal multiplier matrix, 
is shown. In addition the final loadings on the General, the three sub-generals, 
and the six group factors are obtained for the twelve original variables by 
postmultiplying the transpose of the matrix of first-order cluster sums for 
each test by the transpose of the first-order transformation matrix. 


Discussion of Results 


Inspection of the final factor lo 


adings in Table 5 discloses that the six 
group factors possess simple structu 


) re with respect to one another and that 
the three sub-general faetors have this same property among themselves. 
Comparison with the final results obtained by Schmid апа Leiman shows no 
discrepancy greater than plus or minus .0001. 

It has been demonstra 
group factor method to the original correl 
correlation tables at higher orders will 
identical results as the Schmid-Leim 

The chief virtue of t| 


ted that successive applic 


; and the rotation of these 
to orthogonality prior to such rotation, since it uses the cluster sums directly. 


being completely orthogonal. ; 1 
t у ogonal, is easily used 
to get the residual table. It also presents all factors from all CM eae ша 


single table so that the Whole picture can be seen at a glance. Its group 
factors are comparable to the Thurstone primary factors laesi bs the 
present factors are orthogonal. Its Sub-genera] factors bear the same re- 
lationship to Thurstone’s “second-order” factors, : 
in this instance, to a Thurstone “third-order” factor. 
of the factors would be unchanged—with possibly 
the higher order factors since the projections of all 
now explicit on all orders of factors rather 
factors. Since the two types of solution are 
question of whether factors are "really" obliq 
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A GENERALIZATION OF THE BUSH-MOSTELLER MODEL 
WITH SOME SIGNIFICANCE TESTS* 


Many I. НАМАМТАТ 


UNIVERSITY OF CALIFORNIA, BERKELEY 


А generalization of the Bush-Mosteller learning model is proposed in 


connection with two-alternative learning situations with continuous rein- 
forcement. The problem is to test the hypothesis that reward and non- 
reward are equally effective in pon learning. Statistically this reduces 
to testing а hypothesis about the value of a single parameter 0, while a set 
of other parameters remains unspecified. The test presented has the property 
of being asymptotically locally most powerful among all tests of the same 
size and asymptotically similar. The application of the test is illustrated. 


This paper is concerned with testing the effectiveness of reward in 
simple learning situations of the following type. А subject is observed over & 
sequence of trials, on each of which he must choose between two mutually 
exclusive responses, A and B. When he responds by choosing А he receives 
а reward, and when he chooses B he either receives no reward or is punished. 
Experiments indicate whatever choice the subject makes on successive 
trials he eventually learns to make response А. Both à rewarded and an 
unrewarded response may contribute to the learning process, but the two 
experiences may differ in their effectiveness towards that end. The problem 
is to determine from a set of experimental data whether reward or non-reward 
has а greater effect on the subject’s learning of that response. 

A typical experiment in this class would be running а hungry rat through 
a simple T-maze. After going through a short runway, the rat has to choose 
between turning right (response A) or left (response B); the right branch 
leads to a box containing food (reward), the left to an empty box (non- 
reward). These same conditions—the hunger drive and the outcomes of the 


two responses—prevail on all trials. | 
One can also set up the maze with food on one branch and an electric 


shock (punishment) on the other. Or, to take a possible variation out of many, 
one might compare the effectiveness of different degrees of reward by placing 


а large amount of food in one box and a small amount in the other. 
In some of these experiments, there may be doubt as to the exact nature 
to be learned by the subject, whether, for instance, 


* ik xpress m; gratitude to Professors J. Neyman and E. L. Scott, De- 

ызы: өе о of California, for their constant assistance and encourage- 

ment throughout the research that led {о this paper and during its preparation, and to 

Professor F. W. Irwin, Department of Psychology, University of Pennsylvania, for his 
а tions and comments. А " 

many, belt at the American University of Beirut, Beirut, Lebanon. 
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а hungry rat learns to avoid one box because of an electrie shock in it or to 
seek the other because it contains food. Such distinetions are not made here; 
rather, responses А and B are defined in terms of the subject's overt behavior, 
then the effects of the outcomes of A and В are examined. In the case of the 
hungry rat finding a food box on turning right and an electric shock on turning 
left, a right turn is classified as response A and a left turn as г 
The question is whether food or shock is more effective in encouraging the 
rat to make response A on future trials. If а satisfactory answer to this 
question is obtained, it may then be desirable to redefine responses Л and В. 
But the problem of interpretation is left strictly within the domain of the 
experimenter. 

Learning situations of this nature were 
and Mosteller [1] in the application of their 
model presented and used in this paper was a 
Mosteller approach, but allows for more 
transitions. 


esponse В. 


among those treated by Bush 
basic model for learning. The 
ctually suggested by the Bush- 
generality in the trial-to-trial 


The Model 

Assumptions 
The two-alternative learning situ 
sequence of trials on each of which the 
responses, A (the response to be learne 
this subject at the start of tri 


ation described above 
subject тп 


avior of 
al factors 
all combine 
other, This 


acts by making one of the two 


Ў the trial can be observed and 
For instance, the experimenter ma 
the result of the subjects choice on the trial j авт; , where v;l8lifth 


makes response А, zero if he makes response B. 
With every sequence of trials 1, 2, ri e, two other seque 
describing the behavior of the subject over these trials 
the sequence of probabilities Ф%,Ф,- 
and the sequence of outcomes Фу йы s 
Assume that the change in the probability of a subject’s making response 
B from one trial to the next will in general depend on the experience of the 
subject between the two choices; in other words, assume that, for any j the 
probability q;,, depends on т; as well as on q; . In particular, giyen the d 


1 
bability of response B on trial 


Y record 
€ subject 


nces 
may be connected: 
“3% **# , 0f making response В 
UAM tt , wheres, = Loro, i 


of q; and the outcome of the trial j, the pro 
7+1 18 


| 


NK 


ig 


— VP г Д 
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w,.1g; if the subject was not rewarded on trial j, 
din £1; 119; if the subject was rewarded on trial 7, 


where w;,, is a constant depending upon the combination of factors other 
than the presence or absence of reward which influence the change from 
q; дуа, and where 6 is a constant such that 0 < 0 < 1. Since the outcome 
of response А is always reward while the outcome of response В is non- 
reward, this means that 


_ fwg; if the subject chose В on trial j, 
die бш; уа; if the subject chose A on trial j. 


Using the definition of z; , this relation can be written more conveniently 
as 


(1) dia = 0 wai. 


Unless 0 = 1, the probability g;.; depends on whether or not the subject 
was rewarded on the trial j. In fact, 0 < 1 implies that after being rewarded the 
subject has а smaller chance of making response В than after a non-reward 
trial. This finding would support the assertion sometimes made that reward 
has a stronger positive effect on learning. On the other hand, 9 = 1 implies 
that the effects of reward and non-reward on the behavior of the subject 
are identical. 

Iterating relation (1), 


42 = F' Wen , 
Qs = 0"w,q; = F WW , 
“= G+ рлу 7 


апа in general, 
q; = giusti, ... WIH (j = 2, Ses +); 


w may be written for qı , then из for wag , etc., since each of the factors is 
àn unknown parameter. In general, 


ш = W.ctt Wi - 
Now 
gi = puit rote G = 1,2, °°); 
where т is defined as zero, and и; is а constant such that 0 < м; X 1. The 
и; Must remain in the unit interval because 4; is a probability; the 9; are 
taken to be strictly positive to avoid having all probabilities 4;+ь after the 
er what the subject learns. 


trial 4 'o no matt Е 
еее ha 8 is the number of rewarded trials among the 


Since the exponent of 
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first į — 1, 0 < 1 again implies that the occurrence of reward before trial 7 
decreases the probability of response В on that trial. On the other hand, 
6 = 1 implies that оп trial j, 4; has a fixed value и; , no matter how many 
previous trials were rewarded. In the remaining part of this section, a set of 
random variables will be defined with distribution depending on 6. The 
question of whether or not reward has a greater effect than non-reward on 
learning will then reduce to the problem of testing the statistical hypothesis 
that 0 = 1 against the alternative that 0 < 1. 

Suppose that the experiment consists of m consecutive trials for each 
subject and that it is run independently on r different subjects. Define а set 
of rm random variables X;; ( = 1, --- ,r;j = 1, +++, т) by 


Xe usa |! if subject 2 makes response А on trial j, 
ы 0 if subject 2 makes response В on trial j. 


In the same way as above, let 


q;; ^ P [subject 2 makes response В on trial 7 | outcome of previous trials], 


where Р[ Е, | E;] denotes the conditional probability of event E, , given E. 
Then 


Qa = PIX, 20|z4 , + › 2,3-1]. 

Define a set 2 of admissible hypotheses on the distribution of the random 
variables with the following assumptions. 

(i) All subjects have the same initial positive probability on the first 
trial, that is, фа = а > 0 (i =1,---, 7). 

(ii) Subjects act and learn independently of each other, that is, the 
Ху; are completely independent for different $. : 

(ii) Assume 
(2) qi = 0 и; G= lps rje 1, .+ , тп), 
where 0 < 0 € 1,0 < u; € 1, and £o = 0. 


The third assumption states that for subject 2 the probability that 
Х.; = 0 depends on the sum УЖ =. Та, Which is the total number of A re- 
sponses made by that subject before trial j. It also depends on 6 and, for 
each j, on a different parameter и; . However, the и, are the same for all 
subjects. 


Relation to the Bush-Mosteller M odel 


A direct application to this problem of the basic model introduced by 
Bush and Mosteller ([1], ch. 1) results in an expression for q,, identical with 
(2) in form but involving only three instead of m + 1 unknown parameters. 
Bush and Mosteller assume that, for the subject 7 and any pair of probabilities 


* 


Ж -— 


— 2 
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дә and Ф. е transition in probability from q;; to 0:.;+: can be ex- 
pressed in terms of the outcome of the trial j and some constant parameters 
of the learning situation that do not depend on j. In particular, their “Яхед- 
point form" ([1], р. 29) assumes that 

diia = alii + (1 — оу, 


where а and А are constant parameters of the event occurring on the trial j. 
In this learning situation, where response A is always rewarded and response 
B never rewarded, the "fixed-point" parameter ХИ), p. 60) is zero, and the 


basic assumption reduces to the expression 
_ Shai; if subject 2 chose В on trial ј, 
Фан = bo, if subject ¢ chose A on trial j, 
for any j = 2, «++ , m, where 0<0< 1,0 <b < 1, 50 that 0b is their reward 
and b their non-reward parameter. 
Using the random variable X 
the subject 7 is 


‚; already introduced, this probability for 


= 0 bgi; (і-2,--. т). 
(1) іп form, with the constant b replacing 
шуул for all j. Thus, it isa special case of (1) where the trial-to-trial transitions 


in the probability q; are more restricted than in our general model. А 
This iterative relation can again be used to deduce a general expression 


for 4; as in the above section. In this case, 


4: = G bqa 


601 


This relation is identical with 


OF bqis = g+= aa " 


qis = 
and, in general, 

Qi = pete pP gg . 
Assuming that qa = 07 o@=1, ‚ г), which is equivalent to assumption 

at qi = ? 
(1) in the model, 
D 
= 0 а. 


(3) qii — 
4 itional restrietion that 
КР А me as (2), with the addi n tha 
NES aec oe the > Б J ‚ и» аге therefore replaced in this 
шщ = g. The ра 67714. It is possible 


; Н 1 rence 4, 0d, 777» 
special case by a decreasing geometric sequent. ; 
НА зана а on the parameters Ui in the model by assuming that 


s ing it closer to the Bush-Mosteller 
1 eM m , which would bring 1 n | 
vui "n can alt (9), р. 19) that the test proposed in € ее. ейп 
would still hold in this partially restricted case. However, the results given 
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below were obtained without an ordering restriction being imposed upon 
the parameters. 


Testing the Hypothesis that 0 = 1 


Theoretical Considerations 


In order to state the testing problem more precisely, the distribution of 
the random variables under the set © of admissible hypotheses is required. 
The general form for this distribution can be obtained from (2), which gives 
the conditional probability P[X;; = 0 | ха , --- › Vo] for any one of the 
random variables Х;; as 


iz: 


ain 
ВХ чо | Bay ts Zia) =O" uy (1,7 ізі1,::-, т). 


Starting with this relation, one сап verify that the joint probability dis- 
tribution of the rm random variables is the product 


P[X; = z; ; аП?апа j| 9] 


e Па, 


i=l] j=l 


Ш 


where 0 «0 < 1апа0 <u; <1(j=1,--- ‚ т). 

The hypothesis to be tested is the assertion that reward and non-reward 
have identical effects on the behavior of the subject. In terms of the param- 
eters of the distribution, this is the hypothesis, 


H8, 


во that in this case the joint probability distribution (4) reduces to 


E 


) 


^ ae 
() PIX, 225; alliandj| Н] = [[ (0 —w)" vw, 
ізі 


where 0 < u; < 1(j = 1, ,m). 

Notice: that the hypothesis tested, Н, is a composite hypothesis, since 
it does not specify the values of the parameters u, , --- , un . The problem 
is to find a test of this composite hypothesis against the set of alternatives 
that 0 < 9 < 1 ata fixed level of significance a. 

Since the distribution of the random variables X;; which is given in 
(4) and (5) is diserete, it will in general be necessary to use a randomized 
test in order to make the probability of rejecting the hypothesis when true 
equal exactly a. With a randomized test, the points of the sample space 
under consideration are divided into three mutually exclusive sets: w, , ws , 
and әз. Each point represents a possible matrix of observed values x,, and 


- Ка XM «.. 
db c ды 


= 


%> 
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will be referred to briefly as a sample point x. The hypothesis tested is rejected 
whenever the observed sample point z falls within оз and is accepted whenever 
x falls in w . When 2 falls in өз, the hypothesis is either rejected or accepted, 
the decision to reject or accept in such a case being determined, with proba- 
bility specified by the test, by some random process completely independent 
of the experiment producing the observation v, such as the use of а table of 
random numbers. In order to be more precise, the Lehmann-Stein notation 
[3] is used. Define any test T in terms of its critical function у(х), with 
0 < y(x) € 1, where yla) is the probability with which test T' rejects the 
hypothesis M when the point x is observed. Thus « is the set where у(х) is 1, 
while о, is the set where (ж) = 0, and оз is the set where 0 < ya) <1. 
The value of y(x) is determined so that Ely(X)| Н] = а. Here, а is the 
probability with which the test T rejects the hypothesis Н; it is also known 
as the size of T. 

important to consider its 


In addition to the size of the test, it is also ; { 
power. In particular, following the theory of hypothesis testing developed 


by Neyman and Pearson [4], it is desirable that for the composite hypothesis 


Н there is a test Т with the following two properties. 


(i) It is a similar test of size a. In other words, the probability of re- 
jecting the hypothesis Н when @ = 1 is equal to а whatever be the values of 
U, +++, Um in Q. This property can be stated symbolically as 

Ely(X)| Н] = 2 for all possi 
{ is most powerful in the sense that, when 0 < 1, the 
the hypothesis that 0 = 1 be the largest possible 
f the same size. И Н were tested against & simple 


ble values of и; - 


(ii) The similar tes 
probability of rejecting 
among all similar tests 0 
alternative 

K:0-—89, =, 

where 0 < 6’ < 1 and with и; а specified set of values of и; , this property 
could be stated symbolically as 

Му | E] Z Elp) | К}, 
tical function of any other test Tı having property (i). 
milar, Т' possesses property (ii) inde- 
us, then T would be the most 
Q. Such a test is known аз à 
ful similar 
mor 


where y, (X) is the eri 
Now, if in addition to being sin 
pendently of the specified values 0, ui» 77^ ? 


powerful similar test against all alternatives 2 N apd quem 
uniformly most powerful similar test. A uniformly 


test would be desirable in this problem since the yaun a Е А қ > 
are not specified in the alternative against ghi ie PUn against в simple 
other hand, the similar test of H W 


hich is most pow aT 
alternative in 9 depends on the value of any of the parameters spec ed by 
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the alternative, then the test is not most powerful against all the alternatives 
in 9, and there exists no uniformly most powerful similar test of II. 

In the event that no uniformly most powerful similar test exists, one 
might be satisfied with a similar test that is locally best, in the sense that it 
wil be most powerful in detecting small deviations from unity in 0. Thus, 
while still requiring that the test define а rejection rule based on the obser- 
vation z which is independent of the values of ”,- 
sider only “local” alternati 
0 — 1. 

Unfortunately, however, it has been found that neither а similar test 
that is uniformly most powerful against all alternatives in Q nor one that is 
locally best exists ([2], secs. 3.1, 3.2), as such tests appear to depend on the 
values of the unknown parameters. 

Ап alternative approach is possible by means of a method for const; 
ing asymptotic tests for composite hypotheses first given by N 
in the case of a single unknown parameter. The method utilizes an estimate 
of the unknown parameter and, allowing the sample size r to increase in- 
definitely, defines a sequence of critical functions V.(X) such that, as r — œ 
the expected value of V.(X) under the hypothesis tested tends to the pre- 
assigned level a, independently of the unspecified parameter, A test defined 
by such a sequence of critical functions is called asymptotically similar. 

By an extension of Neyman’s method to the case where the distribution 
of the random variables under the hypothesis tested involves more than one 
unknown parameter, it is possible to construct an asymptotically similar 
test of the composite hypothesis И: 0 = 1. Such a test is given in the next 
section and will be referred to as test Tis 


* , Um , one might con- 
ves with біп the neighborhood of the value tested, 


ruct- 
eyman [5] 


An Asymptotically Locally Best Similar Test of H 

Following Neyman, the sequence of critical function 
derived from a set of » independently distributed functions of 
вау ți, +++, £, , through a number of simple transformati 
use of a set of consistent estimates for the unspecified param 


The functions ¢, , ... › f, must satisfy certain general conditi P 
5 A И $ f 
finiteness and differentiability ([2], ch. 3.3), w "teal 


| hich are easily те. However, 
of all the possible sets of functions Tia oe ,%, satisfying those conditions 
one selects a suitable set such that the power function of the resulting asymp- 
totically similar test would have some opti 


| imum local property. For this 
purpose, first consider a modified hypothesis in Q; in particular, the hypoth- 
esis | 


$ ¥,(x) will be 
the observations, 
ons and with the 
etersu, , ... ы s 


Но: ё = 1; T MEDI. 


т , 
where и; is а specified value of u; gei, 


` т). Now look for the Jo 
size a test, say То , of the hypothesis Н 


1 cally best 
о against the alternatives 


^ 
| 
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K:0«1; 4,--,ш. 
That is, in the class of all size о tests of Но against К, the power function of 


the test Го must have numerically the largest slope at 0 = ПА 
Such a test can be easily obtained from the probability distribution 
(4) by application of the Neyman-Pearson Fundamental Lemma [4], and is 


given by 
у(х) = 1 whenever L»k, 
Ша) = 0 whenever L< k, 
0 < у(х) <1 whenever L=k, 
where 


0 
1--%; 


т і-2 


and К is so chosen that EW) Нч = а. . | 
In close relation to L, the set of r functions {; now is defined as 


XU us dy oe . 
(7) fie 5:805 Ула 021,00 
ї-? Fi kel 

= 1 — ù for all j. For each 2, the function f; is seen to depend on 
the unknown parameters 4) ^ т as well as on the set of dew 
$4,770, im, Which may be considered as а single observation on the гап да 
vector X, = (Ха, 9’ X:n). Under the hypothesis tested Н:0 = e. 
joint distribution of the r random vectors X, , c: » Xr is шүп Бү e: jos 
which it is clear that the vectors are independently and identically 


tributed with the probability distribution 


where 0; 


T oin wm 263 nhe 
(8) Р(Х, = zi |» o o0 Н] = Пу @= 1, ) 


ізі 


. „с. а8 defined in (7) must also be mutually 
Ке 


It follows that the functions {з › °° 
independent. » 
ES uid function у.(х) now is derived from the r functions «г Nc 
through the following transformations. 
(i) For each 7, let the standardized value оғ be 
E сас Elk), 
= c (+ :) | : 
alue E(tz and the variance c(t) 
ndardized function is 


2 


As сап be verified, both the expected У cot 
of the function f; are finite, and the resu uag 
vi 


p= 5 za) 

(9) й = = ігі у іші 1 
pu JEN 
>“ (Уы 
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(11) Now introduce the т functions 


Ра = 95. lg P[X, = a; | ty „+++, ; Н] 
for j = 1, --- , m, and let 
_ gi — Ele) 
$i в(Ф;) 
be the standardized value of ø; . Then each $; (j = 1, ... , m), is a function 


of X; and uw, +++ , Uns 
(ii) Also define m other functions R, , --- , Б, by 


R; = E(Z). 


(iv) Finally, for each 2, combine Z; , 4, , --- ; Фе, and В, , 


++, Rm into 
the function 
2; S Ж. Е,ф 
(10) ҚОзуа 55+ ,4Ы) = лә 


1- Хғ 
and take the average Е 
id 

У, = НИЕ 2. Pg, *** уш). 


In terms of the original observations, 


m ici 
Ti; — V; ES c 
м 1 * > v, 22a w) 
мт = CETT 
T 1 Joe T oon 
із 0; kA 


і 


(11) d 


functions introduced above. 


Y, is clearly a function of the unknown parameters, 
observations z;; (2 = 1, ..., =, = 
. an estimate u* for j = 1, ... 


Equation (11) ean be derived from (9) and (10) and the siio of the 


well as of the rm 
: , m). Replacing each и; in Y, by 
,m, gives the corresponding function 


105 
ух = FEIN pur, +++ ,и9, 


whose definition is independent of Ui, **+, Un. A suitable set of estimates 
can be found by the maximum likelihood method, which gives 


[4 


1 4 
21 

2 usd E 
T PX 


as the maximum likelihood estimate of 1 — u; . 
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Therefore take 


\ 
0, 


% 


1 — ut = vf = whenever * с; 


319 


1 
7? = 2 whenever с; = 0. 


'The provision in the latter case is introduced in order to avoid division by 
zero in У*. 

Substituting these estimates for t , ^ ** » “m (orsi, 
simplifying, 


, Um) in (11) and 


яе аео НЕ шей 
х 25; ин Улы - 1 У (т — Dei 
1 al 


(12) Yrs іті 22 Ci k=? T i қ 
T ЕЕ 
Jb p» т=&@ er а) 
T je 6 ға 


where с; = 0 is replaced by 1/7. "WP ) А 
Under the hypothesis H the probability distribution of this function 
Ү* can be shown to approach the normal with zero mean and unit variance 
asr — œ ([2], ch. 3.3). 
Now the test Т, is de 
a level of significance о: and reject thi 
where 


fined as follows. Compute Y* from the data; choose 
e hypothesis Н with probability үу, (2), 


V =1 if Y, > kes 
М9-0 4 Y «he 
о< у. <1 Ï Y, = kes 
and k, is defined by 
$ 1 “ауа dies dora 
NF ——J] С 
Е 
This test has the following properties. 8 a 
Q) Bs deena of Y* in (12), Bly.(X)| Н] is independent of the un 


Specified parameters t ; ‘`` › Um + 
(ii) Since the distribution of У* approaches №(0, 1) under H asr > ®, 


1 ired significance level. 
th пру (X)| H] 2 % the required SIE є 
4 pu 2 the asymptotic distributions of L in (0) and 


Y* in (12) the test T; may be shown to be asymptotically equivalent to 
v 1 
the locally best test То. 


Therefore test T, is asympt 
alternatives К: 0 < 1; t6, °°"? 


ly best test of H against the 


i local Н 
Em totically similar of size a. 


шы, and ASYMP 
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Application of Test T, 


Given is а set of rm observations т; (6 = 1, --- ,r;j = 1, +++ , т), where 
т is the number of subjects in an experiment of m trials, with 


V0 _ Jl if the subject 2 is rewarded on the trial j, 
Фи = оне subject 7 is not rewarded on the trial j. 


To test the hypothesis H: 0 = 1 (reward and non-reward are equally 
effective) against the alternatives that @ < 1 (reward is more effective) at 
the level of significance а, apply the following rule. 

(i) From the data, form the sums 


G = Dai; Gi ге ly 37); 
i=l 
Then c; is the total number of subjects rewarded on the trial j. Let 


g = 


c; whenever с; #0, 
tat 
= 


^ 
Ш 


whenever с; = 0. 
(ii) Compute 
М = 4 > Ші Ж, 


C 1 У in- ie, 


е 1 т Re» Mm 
С” = fh ee dcr — с), 


іс2 С; ка 


апа combine these three quantities to obtain 


Meg 
1 qo med И 
(13) Y = =r 


(iii) From a table of the normal distribution 


function find the 4 
а point, ka , defined by e Upper 


1 bes 
va! ETT oe deg, 
id 


Then reject H with probability y(x), where 


(14) 


У =1 if УЕ, 
(15) JG) =0 if У<Ь,, 
0 < о <1 if У=Е.. 


Т ЧЕИИН g -— ан 
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" E T probability of rejecting the hypothesis H when Y as computed from 
e data is exactly equal to a should be chosen so as to make (ұ(Х)| H] = a. 


Illustration 


The use of the test T, is illustrated on data from an experiment on 


prediction by Е. W. Irwin and Г. Dyckman*, in which the subjects were 20 


women undergraduates. At the start of each trial the subjects were asked 


to predict whether or not a light would go on. The light then went on, thus 
a may be said that a subject who had predicted it would go on was 
rewarded,” or agreed with. Since the light went on every time regardless 
of the subject’s choice, this is a case of continuous reinforcement. 
Table 1 shows the outcome of the experiment in the form of a matrix 


TABLE 1 


Dyckman from a "Hunphroys-type" Experiment 


Data of Е.М. Irwin and L. 


with Continuous Reinforcement * 
үү Trials (J) 
——: 3 4&7 6 9011.1925/14 1530 1628 19 809022 


1 WEWETTIL CCT te ІІ 727 
2 ОЕ 
E Е Е 
4 Zara Alaior ттт 
5 таз аа 
6 аа 
7 2133221111115: 11001111122 
8 2132211111211 EEE ESET aS aS 
9 кооз Е 
10 ктоо аст ЕЕ 
11 1335982121152 EE PIDE RSI ats 
12 ть 
15 2123213130111: TERE ELT aE 
14 гїн р ТЕСЕ 
15 ЕЕ. 
16 :3:5211211911 HH 1H IH 1 i iii 
17 REITE FE 61111) 
18 EEEE TE 
19 ЖҮРЕЕРЕРЕТІНІЗЗЗЗ т as 
20 ыл с үлде» 

8 ЕСТЕЛІГІ eae 


Тоба1 (су) 18 10 14 151 
a zero entry indicates zx,479- 


* An entry of 1 indicates x7 (reward), 


correct 


with an entry of one for à 
т = 20 


of observations for 20 subjects and 22 trials, 1 
n. In this experiment, 


predietion and zero for an incorrect predictio 
and т = 22, For each j G = 1507» 22), 


ізі 
from which are obtained 
А 20 — с; 
l 0-4 74! 


, 
с» 


" 3 
Personal communication from Р. W. Irwin. 
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and 


'Then compute 


20 22 ізі 

М = > P» pum ; 
ie j=2 Cj фл 
1 2 

C = а 2. (22 — е, 


ss =g E icc 7172 
С" = zi. $2 с; Zeo — of | | 


kel 


= 4.186. 
TABLE 2 
Table Illustrating the Computing of Y 1n the Irwin-Dyclman Data 
20-с! 3-1 20 1-1 1 20 34 
3 22-1 — ej 20-6} — i 2 вх (20-х) 2 р xq == 
215%(20-о) 2x04 ми —— ух, вх 
ст үлі i Yn 6) amp ti уух 

ae m Ш’ 

2 2 0 56 9 

5 19 9 11 136 24 458000 

4 18 9 11 255 17 110039 

5 17 8 12 551 19 21575 

6 16 14 6 415 37 275180 

7 15 15 7 506 47 248225 

8 14 14 6 590 64 "166 

9 15 18 2 626 87 4.5714 
10 12 17 5 677 100 +9953 
n 11 18 2 ns ne 5.0924 
12 10 19 1 752 6.1444 
15 9 19 1 751 15 7.5265 
14 в w 5 802 8.5265 
15 7 19 1 821 350 9.4118 
16 é 20 5 eet 1014211 
x s 5 p 222 1111000 
38 3 80 2 840 256 12.4211 
19 3 20 0 820 261 15.0500 
20 2 19 1 FE 281 14.0500 
21 1 19 1 878 249 15.2105 
22 o 20 o 878 502 15.8947 

339 16.9500 
M= 169.2711 
2555.70 1/2 
Z540 = 167,0 сте (28857072 )1/2 = 0,5426 ү = 2.271 


0.5426 7 4.1856 


To test the hypothesis that 0 = Nus, qt dien 
and contradiction, are equally effective) at the level of significance 05, the 
rule as given by (15) is to reject the hypothesis tested if У > | NAE dee 
Within the framework of the model, the result of the mde LT. 


E computations in 'Table 
2 would therefore lead to rejection of the hypothesis that 0 z i. hich 
, 


———— дыни тан с жынын аы" арычы =з T занынын JR e 


Бг 
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implies ассерќапсе of the alternative that reward (making a correct pre- 
diction) had a stronger effect on learning in this experiment. 

Since the above example is given here primarily for purposes of illus- 
trating the computations, it is not necessary to proceed further into the 
interpretations of the statistical test and resulting conclusions. It need hardly 
be emphasized that the validity of the statistical model must be carefully 
considered in each individual experiment before attempting to draw conclu- 


sions from the application of the test. 


Testing Against Other Alternatives 

on under consideration has been whether the 
effect of reward is equal to or greater than that of non-reward. Thus the set 2 
of admissible hypotheses allows 0 to assume values either equal to or less 
than unity, but never greater. However, in some experiments there is a 
question whether non-reward, or perhaps punishment, has a stronger effect 
than reward. To allow for this possibility, assume a larger set of admissible 


hypotheses, say 


Up to now, the questi 


QU < 035 9, 
at the probability 


kei 


qii = [ ui 


where 0, > 1 is defined so th 


remains less than or equal to one for all j. This is achieved by letting 
0, = 1/max и; . In this new set ©, , the results of a preceding section may be 


immediately applied to the case of testing the hypothesis 


Н: 0 = 1 

. Notice that this situation is à mirror 
ove, namely the testing of И: 6 = 1 
will be natural to expect, there- 
f the test Ti of H, given in (15). 
est of М» against alternatives 


against the alternatives 1 < [E 
of the situation originally discussed ab 
against alternatives in which 0<0 < 1. It 
fore, that the test of Н be the exact mirror 0 


In fact, reversi i lities of (13), 2 t 
aet, reversing the inequalr 1 ; у 
with 0 > 1 is ОЛ ЛӨП, Ву considerations of symmetry this test of Н» can 


be shown to have the same optimum asymptotic won т Шеті 
Т, of H against alternatives with 6 < 1. This test of Н» , to be rete 


asi ] ENS P" 2) as 
Т. , is given in terms of its critical function VC ) 


pope В 
(16) gall E a 
T^ L8 dh 


where Y and К. are defined 28 in (13) and (14), respegtikely, 
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A Note on the Use of the Asymptotic Tests in Small Samples 


By definition, the asymptotic test has the property that its size, or the 
probability of its Type I error (rejecting the hypothesis tested when true), 
approaches the preassigned value a as r — œ. In practice, therefore, one has 
at best only an approximate test whose size gets closer and closer to а as the 
number r of subjects in the experiment increases. The question then arises 
as to how large must the number of subjects be in order that the probability 
of Type I error of the test be within some reasonably small interval around о. 

А partial answer to this question can be found in a study of the behavior 
of the asymptotic test T, in some selected cases in which the sample size r 
is small ([2], sec 5). To some extent the approximation seems to depend on 
the sizes of the statistics c; in the data, where c; — Se 2: 0=1,.. 


т), 
80 that 0 < c; < r. For the case r — 10, m — 4, the asymptotic test gave 
fairly good approximations when none of the c; (j = 1, --- , 4) had an extreme 


value like 0, 9 or 10. For a = 05, the 
such cases appeared to lie between .0 
most examples. The approximations became progressively worse w 
more с; (j = 1, --- , 4) were given values near zero or ten. 

As expected, the asymptotic test gives a better approximation to the 
desired size of the test as the number of subjects r increases, There are 
indications of improvements in the approximation as 
in the length m of the experiment. 


actual probability for Type I error in 
1 and .08, being very close to .05 in 


hen one or 


also 
a result of an increase 
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. SOME IMPLICATIONS OF 
THE LOGICAL CALCULUS FOR EMPIRICAL CLASSES 
FOR SOCIAL SCIENCE METHODOLOGY* 


Winuiax C. Ѕснотлї 
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rical classes (СЕС), one of the few 


An exposition of a calculus for empi of t 
blem of constructs and indicators, 


attempts by logicians to deal with the pro 
is presented. 'The CEC provides the groundwork for a formal structure for 
the situation in which individuals have a degree of membership in various 
classes rather than having either membership or nonmembership—a situation 
nearly always true in empirical research. The CEC is presented and its re- 

ts is mentioned. Ап application of the 


lation to various social science сопсер 
CEC model to latent structure analysis (LSA) suggests alternatives to the 
local independence assumption including one called the local scale assumption, 
which has a close relation to a Guttman scale. 


al scientists an article by 
1 classes, and to suggest 
The article was written 
an earlier article 


Following is an attempt to interpret for soci 
Kaplan and Schott [3] on a calculus for empirical 
some relations of this article to social science research. 
by two philosophers. It was intended as а follow-up article to 


by Kaplan [2]. 

The Kaplan and Schott work is 
logicians as the calculus of classes or se 
totality of objects having a certain pro 
belonging to, or being à member of a gi 


the Socialist Party" or “Mendel is ап Elk.’ › 
if he is а social scientist, has felt the inadequacy of the conception of class 


membership to his own problems, 11 inadequacy which rests chiefly in the 
fact that these logical operations require, for their applicability, a certitude 
as to the class membership ог nonmembership of empirical cases, & condition 
seldom achieved in actual research work. Only in trivial cases can it Бе 
said with certainty that “Аза schizophrenic” or «В js upward mobile. 
Kaplan and Schott (hereafter referred to as K-S) set out to construct 
а logical system which is precisely analogous to the conventional po 
(which they call the calculus of sets reserving the term calculus a | asses 
for the new system which they set out to develop) but differs by introducing 


a notion of degree or weight, instead of the certainty, that an individual 
tion of degree or e air was ao BS CU. auae Pte 
Pant Leena оп a project in social scm Bees aah th prior to and during the 
Paul Lazarsfeld for frequent discussions and suggestions pot x ior to ments; Eu 
writing of this paper. Herbert Menzel also contributed valun| à ‹ na d gomme e i ata, 
fessor Abraham Кари also aided Бу. reading the manuscript and otte n 
TNow at the University of California, Berkeley. 
69 


a generalization of what is known to 
ts. A class may be defined as the 
perty. Àn object is spoken of as 
ven class as "Lazarus belongs to 
› Perhaps the reader, especially 
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either does or does not belong to a given class (set). The combination of 
qualities from which his class is to be inferred they call the individual's 
profile. (The terms profile and weight will be given precise definitions in 
a following section.) 

The calculus of empirical classes (hereafter referred to as СЕС) is felt 
to be worth exploring for social scientists for the elucidation of what are 
called constructs, intervening variables, latent classes, ete. The authors 
attempt to develop a logical calculus (formal model) of classes that ap- 
proximate the empirical classes commonly dealt with in research. As such 
it is one of the few attempts by philosophers to deal with this area. The 
calculus of probability is an integral part of the Kaplan-Schott development, 
just as it is more and more proving to be in empirical work. 

The CEC also has implications for the concept of ope 
tion. The probabilistic character of the relation between 
various measures of it is explicated by the CEC. Perhaps 
definition will prove more useful than operational definit 
discussed at greater length in the original article by Kapl 


rational defini- 
à concept and the 
the term indicator 
ion. This point is 
an [2]. 


Our problem сап be phrased: to construct a calculus (an uninterpre 
logical system) which will provide a more adequate explication of classe: 
their scientific use than is afforded by the conventional “calculus of ci 
which we prefer to call the calculus of sets. 

The procedure of the paper is to construct -.. entities which have a 
degree of vagueness characteristic of actual (empirical) classes, but which, 
when this degree is minimal, correspond to precise sets (those classes for 
which every element either is or is not a member). 


ted 
s in 
s" 


Se: 


It should be noted that the attempt is to construct 
logical system not interpreted in terms of ет 
the authors have in mind a specific interpret: 
In this exposition, this interpretation will 
formal structure will not obscure the rele 
fered to social science. 


a calculus, i.c., а 
mpirical correlates, although 
ation, namely, empirical classes. 
frequently be made so that the 
vance of the concepts being of- 


Preliminary Ideas and Terminology 


K-S defined the field (of inquiry) as the set 
a particular inquiry is concerned. In the 
primarily individual persons or groups. 

Articulation is the partitioning or dividing of а field into a set of cate- 
gories, each of which exhaustively divides a set of individuals into mutually 
exclusive sets according to their specific qualities in the categories in question. 
An articulation then is a set of sets (the categories) of exclusive sets (the 
qualities). For example, a population of army officer candidates (the field) 
may be articulated by use of the variables or attributes (categories), ascend- 
ance, intelligence, age, and length of service. Within each category there 


of individuals with which 
Social sciences the individuals are 
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are certain values which can be assigned to each officer (thus dividing them 
ink mutually exclusive sets), e.g., high ascendance, low ascendance; under 
20 years of age, 21 to 25, 26 to 30, over 31 years of age. А similarity should 
be noted between the notion of an articulated field and the notion of property 
space as elaborated by Lazarsfeld and Barton [4]. 
A category must be exhaustive, i. e., it must assign one quality to 
each individual, and since the qualities are mutually exclusive, the category 
assigns only one quality to each individual. C; ategories are what are fre- 
quently called variables, attributes, or dimensions in social science. They would 
of education, dominance, dependence, 


include for people: age, sex, amount 
friendliness, socio-economie status, ete., and for groups: cohesion, democratic, 


productive, satisfaction, ete. 


Some categories may have few qualities and others many, where qualities 


are mutually exclusive sets of particular categories. Qualities in the social 
sciences are often called values of a variable, scale points, discriminable classes 
along a dimension, ete. Examples are: high, low; very much, some, very 
little; higher, same as, lower; agrees; disagrees. In every case they are mutually 
exclusive, i.e., no individual can have more than one quality. Refinements 
of methods of observation can be taken into account in terms of an increase 
in the number of qualities discriminated. 

To clarify the next terms introduced, an example to be used throughout 
this presentation will be constructed. The field is considered to be all army 


officer candidates at a given army post numbering 1,000, the size of the 
alled for simplicity psycholog- 


field. This field will be articulated in two ways, бале 
ical and sociological (see Table 1). (In passing, it may be noted how two 
different articulations of the same field may be constructed, that each cate- 
gory applies to every individual, and that the qualities within each category 
are mutually exclusive.) 

A A profile is a set of qualities, nem 
А profile describes an individual completely Wi" 
inquiry, It corresponds to what in social science 18 often called a complete 
descriptive category, ог typology, oF a point in à property space. In sample 
articulations the following would be examples of profiles, or complete deserip- 


tior divi z Friendly-Low Participator (IFL); 21\ to 
erae use Inn hat Army Man (25HWA). Tt should be 


25-High Sch d 
ool-Working Class s o: 
noted that the нае is the name for the descriplive catego E pm Іш 
qualities; it is not the name for the individuals who күт 2 ee 
indivi i is са ] З 
The set of individuals characterized by а proite aee Го 


i :eoriminati ible wi 
unit cell represents the limits of discrimination роз“, : S 
Пе nrembrera of a unit cell are indistinguishable within the articulation. 


They must be identified as distinct individuals by criteria external to the 


articulation. 
In the examples, all thos 


category in an articulation. 


one from each 
| th respect to a context of 


е officer candidates who are, 6.6. Dominant- 
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TABIE 1 


Two Articulations of the Field "Officer" 


= ———————————————— 
Psychological Articulation 


_ ee a 


Categories Qualities Abbreviations 
Е о а ERE ра ERROR RARUS акынына анысы 
Dominance Dominant - Independent - Submissive D, I, 5 
Friendliness Too friendly - Friendly - Superficially T, F, S, N 


friendly - Not friendly 
Participation High participation - Low participation H, L 
м 


Sociological Articulation 
мМм——— 102020200000. 


Age 18-20, 21-25, 26-30, 31-35, 36-Ю 20,25,30,35,40 

Education Grade School - High School - College G, E, C 

Social Class Lower - Working - Middle - Upper L; Wọ M, U 

Father's Mili- Regular Army Man - Not Army Man А, N 
tary Status 


С 


Too Friendly-High Participators (DTH) constitute a unit cell. Since there are 
no other categories in the artieulation to use for diseriminating the members 
of the unit cell, they are indistinguishable. Criteria external to the 
must be used to distinguish them, e.g. proper names. 

To consider a group of individuals alike in some w: 
a sub-profile is used. This is a profile of а sub-se 
culation, in other words, a set of qualities one from each of some categories 
but not all of an articulation. The classification 20-College-Upper Class 
(20CU) in the sociological articulation constitutes а sub-profile since it is 
a profile of a sub-set of the articulation, namely, age, education, and social 
class. Other examples of sub-profiles are: Submissive-Low Ран овор (SL); 
21 to 25-High School-Middle Class (25HM); Too Friendly (Т). | 

In analogy with the profile, the sub-profile is the name of the descriptive 
category. Cell is the name given to the set of individuals characterized by 
а sub-profile. The cell has the same relation to the sub-profile as the unit 
cell has to the profile. In the above example all of the officer candidates 
who are classified 20CU would constitute the cell corresponding to that 
sub-profile. The cell would consist of all the unit cells corresponding to 
profiles that had the qualities 20, college, and upper class, regardless of 
their qualities in the other categories, hence the cell 20CU consists of the 
unit cells 2POCUA and 20CUN. 


articulation 


ays but not in others 
t of categories of the arti- 


ф- 


j 


етир 
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Elements of the Calculus 


. A basic notion to the development of the calculus of empirical classes 
is that of the indicator. The indicator is defined as a function which co- 
ordinates to every profile some real number between 0 and 1, inclusive. Thus 
to every profile (or point in property-space) there corresponds only one 
real number in the unit interval (between 0 and 1, inclusive), but for a given 
number in this interval there may correspond many profiles. For example, 
the profile DFH may have assigned the number 0.61; DFL, 0.33; SFH, 
0.61; STL, 0.94; ete, with each profile having exactly one number сог- 
responding to it but a given number (e.g., 0.61) may have many profiles in 
correspondence. 

The definition given for indicator illustr: 
logieal development. Indicator is defined stri 
terms such as function, coordinates, number, 
pirical interpretation. As will be seen, an empiric 
suggested. 


Every indicator determines a class, where с 
(each profile and the number assigned to it by the indicator) determined by 


the indicator. In the calculus of sets, a class may be defined as all of those 
individuals with a given property, SLY red. We may state this in CEC terms 
as the set of couples consisting of each red object and the number 1 assigned 
to it by an indicator function and each non-red object in the field and the 
corresponding number 0. The number 1 is taken to mean possession of the 
property red. The CEC concept of “glass” would make this a special case 
of the more general case where each object has to à certain degree the property 
red, so, for example, à scarlet object would be assigned а higher number 
by the class indicator than an orange object, and much higher than a yellow 


object. mn 
The number an indicator assigns а profile is called the weight trom 


the profile to the class specified by the indicator. To illustrate, ad 
are assigned to all possible profiles for the psychological articulation о 


the field of i iry (see Table 2)- | 
Ties d pie -— For an example of a class mer Bare 
army officers (Officer). Define Officer ‘candidate erm Ae "E 
Pletes officer training." Each cell is à and each cel en у t s eus 
of the profile to the clad Officer. The the function which ass g 
Table 2. 
alation between 
to be explicable prim 


ates the formal nature of this 
су with reference to logical 
with no commitment to em- 
al meaning will later be 


lass means the set of couples 


as ' 
profile, 
indicator 15 


a category, & quality, and 
arily in terms of whether 
an articulated or nonarticulated Вей, Tw 
ated by the use of the one category 1 artic- 
4 ` Low Participator) and then we 
ticulation. High Participator could 


It is important to note 
а class. The differences seem 
Or not the class is applied to 
example, suppose a field is articul 
Mpation (qualities: High Participater, 
apply the class Participation 19 the ar 
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TABIE 2 


Profiles and Hypothetical Weights to the Class "Officor" 
for the Psychological Articulation 


Dominance 

Dominant Independent Submissive 
Friendliness Participation Participation Participation 

High Low High Low High Low 
Too Friendly .29 .18 43 433 .20 4 
Friendly .38 432. .89 214 -87 .1h 
Superficially -47 .82 .29 .63 ET .38 

Friendly 

Not Friendly «61 .19 .22 .06 09 52 


then be defined by giving a profile a weight of 0 if it were Low Participator 
and 1 if it were High Participator. However, if the class Popular is used, 
the weights from each profile (High Participator and Low Participator) 
would vary between 0 and 1, since participation and popularity are not 
perfectly related. 

It is obvious that the situation could be reversed by articulating the 
field with the one category Popularity (qualities: Popular, Nonpopular), 
then apply Popular as a class to this new articulation and assign weights 
of 0 or 1 to every profile. Then the indicator specifying the class High Par- 
ticipator would assign weights from each profile between 0 and 1. Thus 
classes are very similar to qualities. The selection of a category and its 
qualities represents an act of choice in which all profiles by definition have 
one and only one quality from each category. Once the field has been articu- 
lated and a class introduced the weight from each profile is determined 
by its indicator function. Hence a quality could be considered a class to 
which each profile has the weight 0 or 1. 

In social science there are similar terms. A category used to articu- 
late the field is often called an independent variable and classes to which 
the individuals are referred are dependent variables. Гог example, indepen- 
dent variables may be religion of respondent which has the qualities, Prot- 
estant, Catholic, Jew, and income, with the qualities, High, Low. The 
dependent variable “votes Democratic" may then be introduced. A weight 


Za 
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would be assig ^ 
bride cp {тош each profile to the dependent variable. The terms 
eris s € are sometimes used in the sense of categor: 
ша дене 4 id n P Re nonobservable variables, predictor and dn 
ables, 2 nanifest and latent variab! imi i 
осот ey ne ables are similar, respectively, to 
Hence, i Y 
, in the example, Officer was à criterion or dependent variable 
6 , 


but an : ' 1 
үп antecedent explanatory variable like father's ambition as the class 
latent variables such as 


cou a 

сЕ have been chosen equally well. Similarly, 

Ө * 2- һе used. The two cardinal points are (4) there is no restriction 
ype of variable included under class or category, (22) the relation between 


class г wor Е) 
and category (or quality) is very close, the difference resting primarily 
d to an articulated or unarticulated 


on the choice of whether the class is applie 


field. 
An individual x is an instance to the degree p of a class, if the indicator 
ber p. Thus the CEC 


reri po assigns to the profile of z the num 
he Mein differs from the set-theoretical explication of classes primarily 
йен e am individual as an instance to some degree or other of the 
mos E "ij than taking some individuals to be instances without qualifi- 
Pops tots others not at all. Classes differ from one another in the degree 
ана. ү in them assigned to the various individuals in the field of 
Hie y. | ais fundamental notion of а class, as opposed to à set, may be 

presented diagrammatically. In Figure 1, à circle represents а set or class 


SET 


EMPIRICAL CLASS 


FIGURE 1 
Diagrammatic Representation of Empirical Class and Traditional Set 
The shaded portion represents the 
a class; for sets this degree is either 


and ee 
rectangles represent individuals. 
a special case of a class. 


d . 

hys ree of membership of an individual in г 

т 1, for classes it varies from 0 to 1. Thus the set is 
Operations 0% Classes 

defined by rules for the combin 

les for combining classes in 


the set calc 
ombination of classes. 


ations of weights. 
the set caleulus, 
‘lize membership 
If by contrast indi- 


Бе е оп classes аге 
idk эл s to this are the rU 
iin product, sum, etc. 4n 

nmembership in a class ОГ e 
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viduals are regarded as having membership to some degree in every class 
there must be a reformulation of class operations in terms of combinations 
of weights. 

The guiding principles in formulating the rules for the combinations 
of weights аге: (i) classes for which the weights are all 0 or 1 are to have 
the properties of sets; (%) the class connectives (sum, product, ete.) in these 
cases are to reduce to the familiar set connectives; (iii) the weights are to 
correspond as closely as possible to probabilities. These conditions lead to 
interesting field characteristics. Their relation to cumulative or Guttman 
scales is discussed below. 

A note on terminology: K-S use p to designate articulations but here 
numbers will be used in the service of simplicity. \ is the symbol for profile, 
and small letters a, b, с designate classes. The notation 


а) WA, а) = p 


states that the weight in articulation 1 from profile А to class a is p. The 
notation K-S use (and followed here) is that of Reichenbach [5] and is read: 
the weight from А to a is p, or, the weight of a, given А. An example: if À 
is the profile 25HLN and a is the class Officer, and articulation 2 is the 
sociologieal articulation, then 


(2) WA, а) = p 


may be read “the weight from the profile 25HLN to the class Officer is р” 
or, using the probability interpretation of weight, “the probability that 
an officer candidate of profile 25HLN will become an army officer is equal 
to p." Feller’s [1] equivalent notation is 


(3) Ў.а] = р. 


Class Product 


The definition of class product will be given first since it is presupposed 
in the definitions of most of the other operations. Here, as in the conventional 
calculus, the product of two classes refers to the joint or overlapping class 
of individuals who have membership in both of the original classes simul- 
taneously, e.g., the product of the classes Officer and Popular refers to all 
those individuals who are popular officers. 


In the new calculus of empirical classes class product is defined as follows 


() fe = a = (VO, о) = min (WO, а), WO, bI), for alla. 
Equation (4) may be read “ће weight from ^ to class с, Where c is the joint 


class of a and b, is equal to the lesser of the two weights (those from А to 
а and from А to b).” 


The product of two classes is defined as that class Whose weight from 


гь. 
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bar profile is the lesser of the weights to the original classes from that 
pronis; Suppose the weights from every profile to two different classes are 
noan, е.р., Officers and Popular, in other words one knows the weight 
s probability) of every possible type of candidate (profile) becoming an 
army officer, and also of being popular among the other candidates. The 
question is, how сап one compute his probability of being both an army 
officer and popular. Г 
In the set caleulus this probability would, in general, be equal to the 
proportion of candidates who were both officers and popular, in other words, 
the proportion of eandidates who had weight equal 1 for both the classes 
Officers and Popular. An individual case is immediately determinable as 
either both weights 1 or not. In CEC the weight of a candidate being both 
an officer and popular is equal to the lesser weight of the individual classes. 
If his chances of being popular are only 0.2, while his chances (interpreted 
weight) of being an officer are 0.7, his likelihood of being both an officer 


and popular is only 0.2. Here it becomes clear that weight cannot in general 


be given the meaning of empirical probability for this definition is very dif- 
bability of the occurrence of a joint 


ferent from the familiar rule for the pro 
class which is the product of the individual probabilities assuming the classes 
аге independent. 
The only condition under which the above definition of the class product 
as а minimum corresponds to the familiar rules for empirical probabilities 
of a joint class is when one class is included in another. Consider the in- 
terpretation of weight as an empirical probability, i.e., the limit of relative 
frequency. If the properties Officer and Popular are independent, the prob- 
ability of an individual being both popular and an officer is 0.14 (0.7 X 0.2), 
considerably less than the 0.2 called for by the CEC definition of class product. 
The CEC definition, when related to the probability interpretation, posits 
а certain degree of dependence between classes, specifically, that one includes 
the other. In other words, the only conditions under which being а popular 
officer has probability 0.2 is when every popular candidate also becomes 
ап officer, i.e., when the probability of a popular candidate being an officer 
is 1.0. If this definition were extended to more classes such that the prob- 
ability of an individual being in all 70 classes exactly equalled its least prob- 
ability for any class, the n classes form a cumulative or Guttman seale. 
Hence, the interpretation of weight as an empirical probability 1s subject 
to considerable restriction. 
i This definition of class pro 
before these can be outlined it is ne 
Class Inclusion 
the other operations should be 
nditions under which 


duct was chosen for specific reasons, but 
cessary to introduce а second operation. 


ning how 


The basie operation determi 
ration defines the со 


defined is class inclusion. This оре 
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FIGURE 2 
Class Inclusion in Set Calculus (a included in b) 


one class is to be included within th 


e other. In the regular set calculus in- 
clusion is defined, 


(5) aCb-yac-af(p, 
which is read "the class a is included in the class b," 
"the class a is equal to the product (joint class) of a 
presents this diagrammatically, 

The problem then becomes one of 
empirical classes, analogous to the d 
that it is equivalent to the set caleulus in the extr 


equals by definition, 
and b." Figure 2 re- 


Hence the definition: Class Inclusion, 


(6) аС В = [WA, а) = WA, ant), forall у. 


Since class product is defined in terms of les 
a is included in b if and only if the weight to 
or equal to that to b. я 

To follow our example, consider the classes Officer апа Becomes Regular 


Ser Weights from each profile, 
а from each profile is less than 


p" ^ 


pur — 
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FIGURE 3 
ical Class Calcul 


Class Inclusion in Empir lus (a included їп b) 


Army Man (Regular). Officer = Regular if for every profile it is more 
probable that a candidate хі аб profile becomes an officer than it is 
that he becomes а, regular. 

In the familiar set calculus, а is included in b means, in effect, “If an 
individual is à member of a he will surely be а member of b." In the new 
CEC, a is included in b is made to mean, in effect, “Insofar as an individual 
is а member of a he will at least as surely be a member of b.” In more precise 
language, the combined result of the definition of class inclusion and class 
product is that if a is included in b, the weight from every individual to a 
must be less than or equal to the weight from that individual to b. 

ar definition of class product 
notion that “Regular 


hy the particul 
is also an Officer” is 


This, indeed, is the reason W 
n the CEC. For, as a result, t 
man that is а Reg 
al class calculus where all the weights 


ome from candidate to Regular are 1 


was adopted i 
is included in Officer if every 
true in the ideal case of the empiric 
from candidate to Officer are 1 and $ 


and some are Zero. 
Class Sum 


Wo, a + WO, y — Wo, a^ 8! 


() = aM 0) 7» [тд = 


{ог ай Л. y 
" The weight from each pro Et 5 
e probabili sum 9 the wel} А 1 
probability calculus) as the duct, this is equivalent 


of the class product. By the definitions of 
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to taking the greater of the original weights 
(8) ПИ, а) or W(,)]. 


In the set calculus the class sum is simply all the individuals who are 
members of (have weight 1 to) either class. This is easily computed by adding 
the individuals who belong to each class and subtraeting those who are 
members of both classes (and therefore who have been counted twice). By 
analogy, the empirical class caleulus would say that the weight of an indi- 
vidual being either an Officer or Popular is the sum of the weight for being 
an Officer and for being Popular minus the weight of being both. But since 
being both (the class product) has а weight equal to the lesser of the two 
original weights, the class sum becomes: greater weight plus lesser weight 
minus lesser weight. Thus the weight for being either an officer or popular 
is equal to whichever weight is higher. The class sum is then defined as 


that class whose weight from each profile is the greater of the weights to the 
original classes to that profile. 


Class Contrary 


(9) [b = —a] =p; О, b) + WA, a) = 1], forall X 


The contrary of a class is defined as the class to which the wi 
each profile differs from unity by the weight to the original cla: 
teresting property of this definition is that it does not satisfy the law of 

. the excluded middle (ie., the law which states that an individual c 
belong to both a class and its contrary), since the weight from c. 
to the class sum of a class and its contrary is only the | 
weights, not their sum (which is one). 
the extreme values of 0 


eight from 
ss. The in- 


annot 
ach profile 
arger of the two 
As the weights to a class approach 
and 1, the weights to the sum of the class and its 
contrary approach 1, until in the limiting case the law of the excluded middle 
is satisfied, 


Class Difference 


In the set calculus the difference between two sets is defin 


| able in terms 
of the set complement, 1.е., 


(10) (a — p) =n: (a Y — p) 


(а minus p equals by definition а and not p). However, the corresponding 
formula in terms of the contrary of a class, a A —b, is not interpretable 
in general as a but not b since its weight from each profile is either that of 
а or of the contrary b, whichever is smaller. Hence class difference is defined 
as that class whose weight from any profile is equal to the difference in the 
weights from that profile to the original classes, respectively, except, that 
it is 0 if that difference is not positive. 
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(1) |с=а— b} 25 ТО, 9 = max [WA, а) — WO, b), 0]], for all à. 


In the limiting case where the class b has weights of only 0 or 1, the class 
contrary is directly analogous to the set complement, and, as in the set 


calculus, а — b = a (Y —b. 

To illustrate the above operations for the psychological articulation, 
hypothetical values of the weights from each profile to the class sum, class 
product, class contrary, and class inclusion will be computed. Only five 
profiles will be used since they shall suffice for this example, (see Table 3). 


TABLE 3 


Hypothetical Values for Profile Weights to Various Class Operations 


ag NES WR 
2 Нм ЕНЕ кк к нен нен ла нн нана кана 
Sun Contrary Difference 


Product 
Profile  W(Ay»a) (А) WO reb) Wapa) У») м(ља-ъ) 
"SEN BENE ANNE Rx 
Al 0.9 0.81 0.81 0.89 0.11 0.08 
^2 0.61 0.03 0.03 0.61 0.39 0.58 
^3 0.53 0-72 0-53 0-72 0.47 0.00 
A 0.32 0.18 0.18 0.32 0.68 0-14 
^5 0.12 0.12 0.12 0.12 0.88 0.00 


NEUEN MEE 


An example of class inclusion is provided by the product and sum where 
the product is included in the sum, since for every profile the weight to 
the product is less than (or equal to) the weight to the sum. = | 
Class product and class difference explicate the application of this 
Calculus to combinations of responses on а questionnaire. For example, assume 
membership in an empirical class is interpreted as giving a positive response 
to an item, and weight from a profile to а class is interpreted as the nominal 
Probability (to distinguish it from the relative frequency probability) of an 
Individual or type (the profile) giving a positive response to a given item 
(the class), Then the nominal probability of a positive response on two 


items (+++) is the class product of the two weights, while the nominal 
; item and а negative response 


Probabili sitive res on the first 
on the m Gan y d жн to the class difference. Thus from the 
Product and the difference the nominal probabilities of all possible kis dea 
Patterns could be computed. Again the isomorphism with the cumulative 
Scale should be noted. 

This completes the fundament 


al definitions which allow for correspon- 
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dence (in the extreme) of this calculus for empirical classes to the usual 
set calculus. The purpose of the calculus so far is to transform the dichotomy 
of membership-nonmembership, basic to the set calculus, into a system with 
the fundamental properties and operations of the set calculus but incorpora- 
ting the notion of weights or probabilities to the relation between an indi- 
vidual and a class. This makes the set calculus a special case of the class 
calculus where the probability of any individual belonging to any class is 
either 0 or 1. 

Detinitions of the four operations allow the development of a caleulus of 
classes which corresponds in many respects to the usual calculus of sets. The 
following set calculus laws hold in this class calculus (with appropriate sub- 
stitution of symbols): the laws of commutativity and associativity for sum 
and product, DeMorgan's theorems, the laws of tautology, the two distri- 
bution laws, the laws of double negation and of transposition and the transi- 
tivity of inclusion. 

One more concept is needed to complete the analogue to the set calculus. 
Define membership to the degree p, to be symbolized by e", 
of an individual to a class when the weight from the profil 
the individual to the class is p. Class analogues can then be 
propositions, for example, the principle of extensionality bec 
(12) la = b] = [Xea = Xe], forall X. 

In other words, for two classes to be с 
have membership to the same de 
Officers and Graduated Candidate 
classes. 

K-S then proceed to develop this calculus to c 
and empirical probabilities, the reference function (the reverse of an indicator, 
namely the nominal probability that if a certain class exists, a certain profile 
will also exist), classes and correlations, and constructs and intervening 
variables. Space does not permit a detailed exposition of these topics but 


the reader is referred to the original article for a very stimulating develop- 
ment. 


as the relation 
e characterizing 
given for many 
omes, 


quivalent, all individuals must 
gree in each class. In our example Army 
School would in most cases be equivalent 


over the topics of nominal 


An Application of the Calculus of Empirical Classes 
to Latent Structure Analysis 


It appears that latent structure analysis (LSA) pursues the same objec- 
tives as CEC, although the problem is approached quite differently. A com- 
parison of one of the results of these two approaches may prove profitable. 
Since both СЕС and LSA are rather elaborate theoretical structures, the 
possible points of comparison are very numerous. One aspect will be selected 
for exploration—the implications of CEC for the assumption of local in- 
dependence in LSA. This is selected because (2) the local independence as- 


sumption is the fundamental опе in LSA, (ii) this point will illustrate the 


EO 


WILLIAM C. SCHUTZ 83 


value of comparin и r s particular 
E the approaches, and (iii) the result of thi i 

S , 8 b 15 par t 

analysis may be of substantive interest. 


CEC and Local Independence 


: Using one interpretation of СЕС, an alternative assumption t 
раз аце is suggested. This assumption, called the ides pri 
[eo а T has the advantage that it allows the items and latent classes of 
нейм ош аз classes and profiles, respectively, in CEC and 
a е falls heir to all of the logical power of the set calculus, The implica- 
ions of this alternative assumption will be explored. 


A Latent Structure Interpretation of the Calculus of Empirical Classes 


Of the possible interpretations that could be made of the CEC, pro- 


files will be interpreted as latent classes, and classes will be interpreted as 
manifest items or observations. Of course, other interpretations of this 
kena logical calculus are possible, e.g., & reversal of the above. This one, 
however, seems to have some interesting implications. The interpretation 
construes individuals as basically divisible into latent classes (profiles) which 
are usually not directly measured but only approximated by inference made 


from observable or manifest data (CEC classes). 
his interpretation is that latent proba- 


" Ап interesting consequence of tl 
bilities become isomorphie with weights (assigned by the indicator function). 
This is convenient because the latent probabilities give the probabilities of 
à member of a given latent class accepting a given indicator. This is the same 
meaning the weights have in the CEC if profiles represent latent classes 
and classes represent items. Similarly, the recruitment probabilities of LSA 
correspond to the reference probabilities of CEC—both being the proba- 


bility of a certain latent class (or profile) given the item response (or class). 


An example should make this clear. 
we For this illustration we shall 
indicated in Table 4, assume eight lat 
psychological articulation, and three dichotomous it 
The cell entries present the probability that an in 
Profile answers items a, b, and с, plus. . 
This same data is represented in standard LSA form, in Table 5. The 
entries іп CEC terms are W (^, а) where A, takes on values of all profiles, 
or latent classes. № is the reference function. ae 
‚То compute the recruitment (LSA) or reference (CEC) probabilities 
divide the weight by the total probability а given item. These values are 
R (a, X) and are given in Table 5. 


construct a very complex example. As 
ent classes (profiles) taken from our 
ems as shown in Table 4. 
dividual with the given 


Class Product and Class Sum 
tween LSA and CEC appears 


Using this int : divergence be 
i erpretation one ivergen veen Са 
With regard to the үне с regarding combination of probabilities. In 
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the elass product, for example, which becomes isomorphie with ГЗА response 
patterns, consider the case of the response pattern +-+ for the latent class 
OHF. LSA would say the probability equals the produet of the individual 
probabilities or .8(.3) = .24, while СЕС would say that probability equals 
the lesser of the two individual probabilities, or .3. The probability of response 
pattern +++ for CEC = .100, for LSA = .024. In virtually every case 
the predictions would differ. Similarly, for class sum where LSA would say 
It is .8 + .3 — .8(.3) = .86 (for ++), CEC equates the probability of hav- 
ing + on either item a or b to the larger, namely, .8. For +++, LSA gives 
8-F.3--.1— 24 — 08 — 03 + 0.24 = .874, while CEC gives simply .8, 
the largest value. 

Гог elass difference, a and not b (response pattern + —) there is also а 


discrepancy. CEC says that the probability of answering + to a, and — to b 
is equal to either the difference between W (a, а) and W ©, b) or zero, which- 
ever is larger, while LSA treats this like any other class produet and would 
simply take the product of W (A, а) and 1 — W (А, В). Thus, for example, 
for latent class CLF, the probability of response pattern 4-- for CEC 
equals .9 — .2 = .7, while LSA puts this probability of +— at .9(.8) = .72. 
For СНХ, 4- — would give by CEC .5 — .6 = 0. Therefore, the probability 
of + — is zero, while for LSA it equals .5(.4) = .20. 

t is possible to 


With the notions of class product and class difference i 
of the probabilities of each response 


compute all values from the CEC scheme қ 
pattern for each profile. In Table 6 we do this for the first two items and 
contrast these values with those obtained by LSA. 


The Local Scale Assumption 


From the above it is clear th 
LSA model in that it does not assume 
that the assumption of local independenc 
class X, and every pair of items 2 and ) 
(13) Жора РЕРТРен 


the СЕС derived model is that 


at the CEC derived model differs from the 
local independence. It will be recalled 
e of LSA states that for every latent 


The assumption being made in 
(14) Pia min [Pi Peal 
(15) pi... = max [Pi — Pi, 01. 


That is, for any two items ? and j, for members of a given latent class, the 


probability of accepting both items (++) equals the pr que E 
the lesser (rather than the product of the probabilities as ‘yt = + = 
probability of accepting item 4; and not accepting желеу, rger. These two 
difference in the two probabilities, or zero, Whichever 15 Бе» 


equations characterize the local scale assumption. 


| 
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The reason for this name stems from the fact that this assumption is 
true when the items form a cumulative scale for the members of a latent 
class. That is, the cireumstance under which the local scale assumption holds 
js that the members of a latent class are all perfect scale types (within usual 
error) on the set of items from which the latent classes are derived. For 
example, consider a five-point cumulative scale and assume the proportion 
of respondents in each scale type (i.e., having the same response pattern) 
is equal. 

In Table 7 are the weights for each scale type for each item in a perfect 


TABLE 7 
Weights for Each Scale Type for Zach Item in а Perfect Scalo 


Itezs 
Scale Туров A B c р Е 
+++++ 1 1 1 1 1 
++++- 1 1 ї 1 о 
+++-- 1 1 1 o o 
++--- È 1 o о o 
S m 9 о o o 
----- 0 0 о о о 
Мова 83 бт 0 3 ат 


scale. To test the first assumption of a local scale, 
(16) Pi; = min [P; ,P,], 


take any two items, say В and D. The local scale assum 
the probability of this class accepting both B and D equ 
which is .33. Computing this probability for the above lat 


ption would вау 
als min [.67, .33] 
ent class, 


(17) 1+1+0+0+0+40)/6 = 2/6 = .33. 
This result will be the same for any two items. To test the second assumption, 
(18) Pi; = max [Р. — P; , 0], 


let us take any two items, say A and Е. The probability of accepting A and 
rejecting E, according to the local scale assumption, equals max [.83 — .17, 0], 
which is .66. The computation of this probability for the above latent class is 


(19) O+1+1+4+1+1+40)/6 = 4/6 = 66. 


WILLIAM C. SCHUTZ 87 


The zero term in the equation covers the case of nonscale types, that is the 
probability of + on С and — on B, a —+ response, is negative, therefore 
zero. This is the formal statement that there are no nonscale types, or that 
the class includes only those individuals who are scale types. Hence the 
scale-type latent class fulfills the requirements of the local scale assumption. 
It is likely that this is the only type of model that does, though that proof 
has not yet been worked out. 

This assumption, which leads to a set of accounting equations, will 
not be pursued. The suggestion has been made by Lazarsfeld that this develop- 
ment could be pursued by studying the distribution of error. All that can 
be said at this time is that this appears to be a promising lead. The main 
point to be stressed here however is that LSA is based on the local independ- 
ence assumption, but the local scale assumption and in general the con- 
sideration of other assumptions might render LSA more flexible and lead 


to different latent classes that may be more appropriate in some investigations. 
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NOTE ON фф... 


Еруавр E. Сокетох 
UNIVERSITY OF TENNESSEE 


Formulas are given for a descriptive statistic related to the fourfold- 
point correlation but having always-attainable limits + 1 


The fourfold-point correlation coefficient, $, is derived from а table such 
as 


- E 4 

+ Pı — pie Piz Di 

F 

= g= ү + pis Ps — Pe Q 
92 P2 1 


where p, , (hi; P2 , and q are the marginal proportions, and р,» is the joint 
positive proportion. Then 
(1) pw Di — Dip. 

V 2141924: 
" % 
15 the product-moment correlation between the dichotomous varieties X and 
Y. 


It is well known that 9 can equal +1 only if p, = p, = 5, and that it 
сап equal —1 only if Pı = Фф = .5 ([1], р. 324: [2], р. 342). Hence $ has 
limits +1 only if all marginal proportions are .5. 

In order to obtain à descriptive statistic having always-attainable limits 


(2) Ф en Раз — Dip. 

max D — pip: 
This formula applies if ф is positive. But for ф negative, mas Should be the 
absolute value of the largest negative correlation consistent with the given 
marginals; this will occur when cither Piz OT G2 — pi + Piz is 0, and the 
denominator of $/max is then p, p, minus the smallest possible value of pi; . 
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i is simply pip. . If pi; cannot 

the value 0, the denominator is simp’ s 
ааа а — Pr de pi; can, in which case the smallest possible value 
of piz is Pi — 9. For Ф negative, therefore, 


М. v. Ті — Pio 
(8) dus Ф: — (Pr — а)” 
and the second term іп the denominator is subtracted from pip, only if 
m- 


More convenient computing formulas are obtained if one multiplies 
numerator and denominator of (2) and (3) by N*: 


NS. — 816 an 
(4) = = WS SS. ,Ф positive; 


т Ф NS, - 5,6, 
P ыы SiS; — М + 5, 0)'® "Үе 


Неге S, and S, are the positive marginal frequencies, S,» the joint positive 
frequency, S’ = S, or S, , whichever is smaller, and the second term of the 
denominator of (5) is subtracted from S,S, only if S, + 8, > М. 

For the alternative table,* 


Se dr 

Е а b Si 
Y 

= c 4 N- 8, 


NEAS 8:0 М 


the numerators of (4) and (5) are equal to bc — ad. F iti i 
куысы се NR ad. For ф positive, either a 


(6) $ _ bc — ad 


Du Weg 9 positive, 


and b’ and c' are obtained from a fourfold having the same marginals but 


with a or d (whichever is smaller) replaced by 0. For $ negative, either b or c 
must be 0 when ¢ takes its maximum negative value, so t 


(7) $ _ bc— ad 


T vdd ‚ф negative, 


and a’ and d' are obtained from a fourfold havin, i 
à à i 5 the same mar 
with b or c (whichever is smaller) replaced by 0. poe 
The limitation of range of the ф coefficient is i 
у а special ease. ТЬ 
moment correlation always has attainable limits + 1 if both ii 
are 
*This alternative development is based on an editorial Suggestion. 
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continuous, or more generally if no two individuals have identical scores on 
either variate. With grouped or discrete distributions, its upper limit is + 1 
only if the two marginal distributions are identical, its lower limit is — 1 
only if one marginal distribution is identical with the other reversed, and its 
two limits are = 1 only if both marginal distributions are symmetrical and 
identical. 
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BOOK REVIEW 


Сира В. Соомвз AND Rrcuanp С. Kao. Nonmetric Factor Analysis. Ann Arbor: Univer- 
sity of Michigan Engineering Research Institute Bulletin No. 38, 1955. Pp. vii + 63. 


Following a brief description of Coombs' theory of data, this slender technical mono- 
graph treats two classes of models for the resolution of behavior into components. These 
models, conjunctive-disjunctive and compensatory, are intended for scaling dichotomous 
monotone items collected by the method of single stimuli. The models are not competitive 
with Guttman scalogram analysis but are offered as multidimensional formulations which 


might be appropriate when unidimensional solutions are not obtained. A general axiomatic 
, and theorems specific to each model are derived зера- 


basis for the approach is presented 
d throughout the mathe- 


rately. Some very helpful interpretive material is intersperse 
matical development. 
In the conjunctive model success on 


the relevant dimensions; in the disjunctive то 
one of the dimensions involved. The conjunctive and disjunctive models, although psycho- 


logically distinct, are shown to be formally equivalent and are treated simultaneously 
throughout the development. In the compensatory model, which underlies multiple-re- 
gression theory and multiple-factor analysis, a given performance may be achieved by 
many different combinations of components: an excess in one attribute may compensate 
for a shortage in another. The logic of the models, then, would call for the use of multiple 
cutting scores for a conjunctive domain and multiple regression for a compensatory domain. 

The general model is constructed by representing both individuals and stimuli as 
vectors in a multidimensional space, called the genotypic space. An individual's response 
to а stimulus is treated formally as an order relation on the corresponding individual veetor 
and stimulus vector. Theorems are derived to generate а calculus for recovering the under- 
lying genotypic structure from the data. In the conjunctive model the measures in the 
Eenotypie space are assumed to be elements of only an ordinal scale, and the analysis 
recovers the several components at the level of partial orders or simple orders, А method 
is also described for estimating a lower bound of the dimensionality. 

In the compensatory model the genotypic structure is strengthened to the » Ve 
real numbers, but the additional assumptions required to recover the elements at E à 
level are not made, so that factor loadings are obtained on an ordinal seale. Both an indi- 
vidual compensatory and a stimulus compensatory model are constructed, барона оп 
Whether the linear function which weights the several components 18 rien хаа оі ап 
individual and independent of ће stimuli, or vice versa. Only the кале ч е дй. 
pensatory model have been formulated. Procedures have been developed on 4 or ү he о 
case of two dimensions, and the information recovered in the analysis is = Ж e ES 
factor loadings (cosine of the angle between vector and coordinate axis), which does 


correspond to the order of the projections. . i А 

These oh ante а. several problems which are Lara m ee 
Kao in their discussion section. The problem of 2 = еч т rue == у 
deterministic models, and no procedures are offered for its trea um еа ыы: 
aware, however, that 4... the data will contain error, and os li LO р pee ай ia tn 
zations will have to be constructed 0 me iin on ra Milholland has kam that 
arises with respect to uniqueness in handling incomplete data; Mi i | aie шаа 
alternative solutions can be obtained in the conjunctive-disjunctive m : : - e RE а 
important concern involves criteria for choosing between m mo ac voleur tier 
data. The authors suggest that an appropriate model кері таё 
(фал an inappropriate өше sad тойа EE ее precise formulation of а 

everal advantages may accrue from 3 
93 


a task requires a certain minimum on each of 
del success requires a minimum on any 


error is not consic 
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measurement model, not the least of which is the clarity gained from an explicit statement 
of assumptions and inferences. For example, the precise formalization of a model might 
stimulate theory construction in measurement by providing the foundation for further 
generalizations and by making it possible to compare the exact properties of different 
methods. A more important contribution is sometimes made, however, in that the models 
underlying various measurement methods frequently have implications for theorizing in 
psychology. Some such implications, a few of which are briefly illustrated in the Intro- 


duction, are attendant upon the distinctions made by Coombs and Kao in formulating 


the models of Nonmetric Factor Analysis. These implications may be derived in spite of 
the fact that statistical machine 


ry has not yet been developed for practical applications 
of the models. 


The elegance and precision gained through the axiomatic presentation has an ac- 
companying difficulty of reading, so that the audience for this technical 


n monograph seems 
limited to measurement specialists interested in the theory of models, 


SaMUEL Messick 
Educational Testing Service 
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ON METHODS IN THE ANALYSIS OF PROFILE DATA 


SAMUEL W. GREENHOUSE AND SEYMOUR GEISSER* 


NATIONAL INSTITUTE ог MENTAL HEALTH 


This paper is concerned with methods for analyzing quantitative, non- 
categorical profile data, e.g., а battery of tests given to individuals in one or 
more groups. It is assumed that the variables have a multinormal distribution 
with an г EM variance-covariance matrix. Approximate procedures based 
on classical analysis of variance are presented, including an adjustment to 
the degrees of freedom resulting in conservative Ё tests. These can he applied 
to the case where the variance-covariance ma 


trices differ from group to 
group. In addition, exact generalized multivariate analysis met! ods are 
discussed. Examples are given illustrating both techniques. 


Much research in the social sciences is of the multivariate type; multiple 
observations are made on individuals who have been sampled from one or 
more populations. In particular, when the observations are in the form of a 
battery of tests or a set of items, there is the problem of profile analysis, 
est for differences in the levels and in the shapes 
of the group profiles. If the variables being observed are assigned to columns 
and the individuals to rows, the resulting matrix of observations is very 
Suggestive of the data usually analyzed by analysis of variance. Furthermore, 
since the rows are random and the columns can be considered in almost all 


instances as fixed, the appropriate model is the mixed model. | 
Пу computed ratios of mean 


As is well known, in order that the usual: os of mi 
Squares in this model [7, 14, 16] be exactly distributed as the F distribution, 
ddition to being normally dis- 


it is necessary that columns (variables), in a 


tributed, have equal variances and be mutually independent or, at most, 
assumptions seem much too restrictive. 


have equal correlations. But these s 
In ба investigations, it is unrealistic to assume that three orsmore tests, 
items, or treatment schedules have the same pairwise correlations or that 
they have the same variances. It seemed obvious, therefore, that this problem 
of multiple observations should be considered in its per кеа, 
namely, that an individual vector tı , t2, 777,4» 18 sampled тоша p-variate 
covariance matrix. 


normal distribution with an arbitrary variance- А 
Exact procedures for analyzing data of this type Las aet асығы 


Some time and are usually referred to a$ the generalized aide жүнін 
of variance [1, 10, 12, 13, 17]. These, however, require rene gan ome 
y ЕД , 2 
computations than that demanded by the arithmetic 0 4 : а 
*We are indebted to Mrs Norma French for performing all the calculations appearing 
е! . z 


in this paper. 


Wherein it is customary to t 
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variance. Furthermore, an analysis of variance approach permits the analysis 
of a set of data which cannot be handled by multivariate procedures, namely, 
the case where n, the number of random vectors, is less than p, the number 
of variables. Although these multivariate methods are discussed subsequently 
and an example is given for the case of two groups, our main purpose is to 
utilize the simpler, and more familiar, conventional univariate analysis of 
variance techniques under the more general assumptions. Our results concern- 
ing the approximate distributions of the F statistics are based upon the work 
of Box [5, 6] with regard to one group and its extension, by Geisser and 
Greenhouse [8], to several groups. In addition, the latter have found certain 
adjustments to the approximate tests leading to conservative tests which 
can be used, when the group sample sizes are the same, in the case of unequal 
variance-covariance matrices among the groups. 

It is of interest that Block, Levine, and McN 
concerned with the application of the analysis 
problem. They presented F tests for testing th 
(columns) means, the homogeneity of over-all group means (profile levels) 
and the equality of profile shapes. However, they assumed equal variance, 
among the variables and, since they imply that the F tests are exact, it 
can only be inferred that they also assumed the variables to be independent 
or equally correlated. 


emar [2] were also primarily 
of variance to the profile 
e homogeneity of variable 


The Problem 


Our notation is almost identical to that used by Block, Levine, and 
McNemar. Let p tests, z; , 25 , -+ ; Tp , be given to each of n, individuals 
(k = 1,2, +++ , g) in each of д established groups. Assume both the p tests 
and the g groups to be fixed, i.e., they are not random elem 
from larger populations. This model, which fixes interest i 
groups under study, conforms to many experimental situ: 
practice. The totality of Np observed scores (V = У) 
according to the scheme at the top of the next page. 

An individual 2 in group k has the profile 


ents sampled 
n the tests and 
ations met with in 
i-i 4) can be classified 


i = 1, ++- ‚ть individuals in group k 
(Фо) Фик)» jv) Jj =1,-++,p variables 
k= 1,+-- ‚0 groups. 


And the group profile for group k, say, is represented by 


(2. s 2.23 TEST ғы). 
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ol Tests 
II ee TEn 
Grou 
р Ind. ЕТТІ 
1 
1 тіп Тіп Тірі 
т ап Tu лур! 
oe — 
Means: Gr. 1 Bur. Eng cA 2 
k 1 Tuk Tijk Тірь 
2 Tiik Tijk Tipk 
ПЕ ТвыЕ Хпјк Tnpk 
Moa E 
Means: Gr. k Фак £k E.pk Ek 
WE noe MP 
0 d Фо Tijo Тірг 
Ng Ingo Tn, io Та,р2 
Means: Gr. 9 Т істе Е 229 
£g Za. 5... 


Means: All Groups 21. 


profile is à random 


h an arbitrary variance- 


vector sampled from a 


Assume that each individual : L 
covariance matrix, 


p-variate normal distribution wit 


2 . 2 
сі різбіба `` p15010» бі 012 ір 
2 а, с 
х = | 2120102 оз 2445 раб, [ecl а е 
* 3 Оло ОВ р. ЫС. 


9» 
Also assume that the р variables have the same metri 
to give meaning to the question of whether the group pro 


Р1рб10» — P25020» 
c. This is necessary 


files have the same 
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shape, and not because of any statistical considerations. This restriction 
results in no loss of generality if there already exists а large body of data 
on these p tests so that the standard deviations can be assumed known. 
For in this instance, equal metrics can be obtained by standardization of the 
p test scores. 

The questions that are most often asked in profile analysis are: 

G) Are the groups on the same level, i.e., do the groups arise from popu- 
lations having the same group means, namely, E(z.,) = E(z,,) =... = 
E(@..,) , where E denotes the expectation? 

(ii) Do the groups have the same shape, i.e., do the groups arise from 
populations having parallel group profiles? 

Another question that may be asked of these data, although not too 
frequently in profile analysis, is whether the p tests have the same means. 

With regard to the question on shape, it becomes necessary to define a 
statistic which reflects the concept of equally shaped group profiles. In 
a larger sense, profiles having the same shape can be considered to be parallel 
curves. As Box [4] and Block, Levine, and MeNemar [2] point out, parallelism 
сап be measured by the group-test interaction mean square. That is, if the 
curves are parallel, the group-test interaction should be zero and the mean 
Square should not differ signifieantly from an &ppropriate error mean square, 
If, on the other hand, the curves have different shapes, the interaction mean 
square should be significantly greater than the error mean square. 

This is made clear by reference to two group profiles: 


Vau DE TEE ‚ зл 
апа 


$42,225, Я. 
Denote the corresponding differences between group me 
didz, ,d,. 
If the two profiles are parallel it is clear that d = а, = q. 


= бош эн T 
On the other hand, if d, = d; = d, = --- = d, › then the two profiles must 


be parallel. Hence, a necessary and sufficient condition that the two group 
profiles possess the same shape is that 


ans for each test by 


д =@ =, =... а 


But the equality of these differences is exactly whatis meant by no interaetion 
between groups and tests, and the extent to which these differences are 
unequal corresponds to the existence of the group-test interaction. "Therefore 


a test of the group-test interaction is also a test o whether group profiles 
have the same shape. - 


T'ests of Significance In the Mixed Model 


If the p test scores have equal variances and are independent (or, at 
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most, are equally correlated in pairs), so that 


a Wy х-- 70 
%-1|0 6" sse 0 
0 0 о? 
ог 
1 р p 
у = |р 1 pli 
рр 1 


= the given scheme constitutes the classical mixed model for g samples, 
th. proportionate numbers of observations among the samples. The appro- 
priate analysis of variance breakdown is shown in Table 1. The analysis 
under either of the above assumptions on the covariance matrix follows 
along classical lines. The Fi , P»; and F, statistics used to test hypotheses 
of homogeneity of test (variable) means, of group means (level) and the 
nonexistence of a group-test interaction (equal shapes of group profiles), 


respectively, are exact. 

If, on the other hand, the validity of these two models is suspect, on 
the basis either of prior evidence ог of a statistical test, the given Г ratios 
are not distributed like the tabulated F distribution. In this situation where 
the covariance matrix is assumed to be arbitrary and given by 3, Roy [13], 


Rao [12], and others have approached the problem through the multivariate 
f interest, and possibly of considerable 


analysis of variance. However, it is o | 
practical importance, to investigate the distribution of the computed F 
Statistics. 

‘or Arbitrary Covariance Matrix 

veral groups Box's work 
are each independent 
that, under the 


T'ests of Significance f 

Geisser and Greenhouse [8], in extending to 86 

[5, 6] relative to one group, have shown that Q, and Qs 
of Q; , and Q, is independent of Q, . They have also shown 
null hypothesis, 


EQ, = A, say, 20) = ( HQ) = (N — 94, 


gc 1)A, 
and 
say, E(Qs) = (N = 9B. 


nd the expectations of the mean 


EQ) = (g — DP; 
Table 2 gives the mean square (MS.) а 
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square (E.M.S.) for each of the five sources of variation in the analysis 


of variance. 
From the results presented in Tables 1 and 2, it follows that each of the 
three F ratios, F, , Ғ,, and F; , is a ratio-of two independent mean squares 


TABLE 2 


Analysis of Variance 


M.S. 


Tests (p-1)^Q @ | 

Groups (8-1) ^q Д 

Individuals within Groups (м-в) ©; ^ 

Groups x Tests (51) (8-1) 9, Sag 
(p-1) lA 


* LI E ji 
Individuals x Tests (p-1) Қа-в) 9 


within Groups 


“наз. ee ee 


г the null hypothesis. Making use of the 
orms involved in the three Ғ statistics is 


exactly distributed like a linear sum of independent х? variables with the 
6.1, Box [5, 6]), #1 13 approximately dis- 


same degrees of freedom (theorem | аде! 
tributed like F((p — le, @ — D(N — g)d, Ёз 18 approximately distributed 
like F((p — D(g — De (p — DO — д) е, and Г, 15 exactly distributed like 
Р(0 — 1, N — g), where 
_ в ууф — DGZes. — pre; + 99.) 

гіх Х, б, is the mean of the diagonal terms, 


(or tth column), and 2.. is the grand mean. 
‘ance matrix, which must be 


with the same expectations unde 
fact that each of the quadratic 1 


e = p (ди 
Tia ате the elements of the mat 
m 13 the mean of the ИВ row 
us, the effect of the arbitrary variance-covariane 
the same from group to group, is to assess the significance of ДЕ. Т н F, 
statistics in the ordinary tabulated Е distribution but with reduce ds 
of freedom. The Ғ, test on group means, it will be noted, ies ps ange 
from the standard F test since it results from & one-way analysis oi variance 
With all observations having the 84 


me variance. | | A 
The reduction in the degrees of freedom for this approximate test is a 
function of the elements of the 


population variance-covariance matrix. 

e estimated from 
This is almost never known, and therefore € will E ӨСЕ ДС 
the sample variances and covariances. Hane lved is unknown. Hence 
mated є on the approximate ү distributions involve 
unless the variance-covariance mM 


atrix 18 estimated with a large number A 
j 1 j ested. 
degrees of freedom, use of the conservative test given below 1s sugg 
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A Conservative Test 


The preceding approximate procedure requires some computations on 
the elements of a known variance-covariance matrix, In many profile problems, 
the number of tests may be as high as 50 if not more. This results in a 50 X 50 
matrix, necessitating some laborious arithmetic. Е urthermore, in almost all 
problems variances and covariances are unknown and the extent to which 
є is changed by using sample estimates has not been investigated. As а 
result it is useful to obtain a lower bound on e; it can be shown that 

аза, 
Ф--1 
This minimum value of е is independent of the elements of the variance- 
covariance matrix. 


With this new correction to the degrees of freedom, the F, and F, statistics 
are now judged for significance by entering the tabulated Р distribution 
with 1 and N — g degrees of freedom and with 9 — land 
freedom respectively. These tests are called conservative gi 


value of е gives the maximum reduction in degress of free 


N — g degrees of 
nee the minimum 
dom. 


An Example 
Five groups of mothers, classified into their grou 
external criteria, were given a maternal 
23 scales. For purposes of this illustrati 
selected. Thus p = 6, 9 = 5, and М = 
means are given in Table 3. 
The five variance-covariance matrices were firs 
The likelihood ratio test, the multivariate analo 


PS according to some 
attitude questionnaire containing 
on, six of these scales have been 
128. The group profiles and group 


t tested for homogeneity, 
gue of Bartlett's test for 


TABLE 3 


Mean Profiles for Five Groups of Mothers on Selected Scales ofa 
Maternal Attitude Questionnairex 


Groups Mothers 1 3 6 9 13 i Group 


Mean 
B 13 17.92 13.85 17.23 14.00 12.23 17,38 15.44 
с 15 18.07 11.60 14.13 8.93 8.27 17,73 13.26 
р 32 16.75 14.47 15.41 11.78 9.91 15.94 11.08 
Е 9 18.33 10.78 13.89 15. 12.1 


18.78 14.72 
All Groups 128 17.35 12.20 14.34 11,79 10.05 16.27 13.65 
*We are indebted to Dr. Richard Q. Bell of the Laboratory of Ps hi 


9 use part of his data for this 
example. 
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h ity rari 
сш of variances, can be found in Вох [3, 4]. (Kullback [11] deriv 
quivalent test through information theory.) The test statistic is ү 


М = М log, | S| — Уп: log. | S: | = 112.6565. 


In the i i 

idi PX | 8 | is the determinant of the pooled variance-covariance 

= d an | S; | is the determinant of the sample variance-covariance 
atrix in the ith group. Now compute 


2? 2 
А NUS LM Lia T 
dd Iu IE а т = .17012, 


ала 


f, = pp + DU - 0 = 85 
a ; рН . 
а! enter (1— А )M — 93.4 in the x’ distribution with 84 degrees of freedom. 
miti the probability of getting this value of х or larger is fairly high, the 
hypothesis of equal variance-covariance matrices is not rejected. 
An estimate of the matrix E is given by the pooled variance-covariance 


matrix 
3.100 101 —.279 —.083 = .009 1.557 
101 5.780 1.013 — 114 —1.014 089 
ы —.279 1.013 5.560 1.039 1.866 —.169 |. 
—.033  —.114 1.039 5.600 3.080 .258 


3.080 6.820 .222 
258 (222 5.170. 


al variances and equal 
niform variance- 


-.009 —1.014 1.366 
1.557 .039 —.169 


Consider now whether the hypothesis of equ 
covariances is consistent with S. The best estimate of the ш 
covariance matrix under this hypothesis is given by 
A67 с 467 


5.3888 
т 5338 e 467 
i= 3 5 ; i ` 
A07 467 5.338. f 
in S and the 


Where the diagonal element is an average of the 6 variances 
ovariances 1n S. The reason 


Ганга is an average of the 15[4р(р — D] cova 5 
or testing this hypothesis is that if S, is consistent with the data then classical 
analysis of variance procedur ble. The test used is again а 


кке es are applica ae 
kelihood ratio test, also given bY Box [3, 4]. The test statistic 18 
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4 ЛІ 10,80042 _ < o; 
M = —(N — 9 log, | 8; | = —123 log. 21,040.49 81.956, 


where (N — g) = 123 is the degrees of freedom entering into the computation 
of any element in S or S, . Now compute 


Ё Ф 10р 3)  - 
A: = B= ale — Nort pw = +01887, 
and 


h = @ +p — 4/2 = 19, 


and enter (1 — A,)M = 80.4in the x^ tables with fı = 19 degrees of freedom. 
The probability of this result is well below 001; the hypothesis of equal 
variances and equal covariances must be rejected. 


The analysis of variance yields the numerical results of Table 4. 


TABLE l 
Analysis of Variance 


Source d.f. 55 


M.S. F 
Tests 5 5092.56 
Groups 4 509.12 127.28 Fy = 16.51 
Individuals within Groups 123 948.11 т.т 
Groups x Tests 20 644.74 32.24 Е, = 6.63 
di pies 615 2991.01 4.86 


Of primary interest is the test of the homo: 


is a test for the existence of the group- 
enter the F, value in the F table with ( 


geneity of group profiles, which 
test interaction. For this purpose 
9 — 1)(р — 1)eand (N — g)(p — le 
Brees: | dom. From the previous formula, and 
1X, є1з estimated to be ‘8194. Therefore the effective 
degrees of freedom are 16 and 503. The observed p, = 6.63 is greater than 
m. One therefore rejects 
( the mean profiles differ 
in shape from group to group. 
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as the approximate test, namely, the probability that the group profiles differ 
in shape due to chance is less than .001. 
'The groups clearly differ with regard to levels as can be seen from the 
very large F, value. 
Other Procedures 


'The foregoing procedures present approximate and conservative tests 
of significance resulting from the analysis of variance utilizing readily available 
tables of the Г distribution. As mentioned earlier there are available exact 
ariate analysis of variance. These procedures lead 
to exact tests of the general hypothesis in multivariate analysis of the equality 
of vector means among g populations and of the existence of the group-item 
or group-test interaction of interest in profile analysis. However, all of these 
procedures require laborious computations involving the inversion of (р X р) 
matrices (p equal to the number of tests or items) and the computation of 
latent roots or the evaluation of determinants. A further complication is 
the lack of tabled probability values for the appropriate test statistics. 
Recently, however, distribution tables have appeared relating to the approach 
of multivariate analysis initially taken by Roy [13]. Under this view, the 
distribution of the test statistic is dependent upon the distribution of the 
maximum characteristic root of certain matrices. The most comprehensive 
tables or charts thus far available are those given by Heck [9]. Heck, inciden- 
tally, specifically considers the problem of profile analysis. — 

The case for two groups will be developed in some detail to illustrate the 
principles involved and then the extension to g groups as уор by Heck 
will be summarized briefly. The former situation leads to Hotelling s general- 
ized T? statistic and is implied in the literature on multivariate analysis. 

In the previous notation, tiir is an observation on item j for d 
i in group k, and 2. is the mean of character j in group ЁК. О eee о 
subscripts here is = 1,2;j = b 2,...,p;andi = 1,2,5: Ms s before, 
assume that the random vector 220) = (бақ) там) 18 N (поз, 2), that 
i ; ivari 1 distribution 1n population Ё 
is, the р variables have & multivariate norma. : Eus 
with mean vector и = Сао ны) ала ас a dm 
= which is common to the 9 populations. The hypothesis to be tested tor 
9 = 2is 


procedures in the multiv: 


pac Ш > Ber — ghee d ры — Meas 
Transform the p variates in 2 to — 1 variates in у as follows (see [1], 


рр. 110-112 and [12], рр. 239-244): 
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such that Уу, c, = 0. The matrix C, subject to the restriction, can be 
perfectly arbitrary. For example, 


1 —1 0 0 
—1 0 
с 1 0 
1 0 0... —] 
subtracts 2» , +++ , æ, from the first variate resulting in У = Ta —– 0, 0 = 
Mi = у= а = 1 — 2,. Or, 
psi -p p oe si a 
on} -1 Pot Дд ms 24 =t] 
-— —1 =] p=] =] 


each of the р variates their mean @ = (1/p) 

p X 1 „ш 2-1 — 2. Using the first 

7 У-у) is multivariate 

1775, None), Nin = Hir изъ, and variance- 

covariance matrix CZC' where the prime denotes the transpose of а matrix. 
After transforming the p a-v: 


i ariates into the P — 1 y-variates for each 
of then = n, + п» individuals, the group means in the y's are 


3-і 2) resulüng in y, = 


Vai а yes › 9. опа 


Gare s Taz gy Pg 9. 6-1» , 
and the pooled sample variance-covariance matrix in the ys, W = (шы) 
where : 
шы 


1 m 
i n, — 2 (5 Yin — 9.) (у: — F.a) 


P >. (у: — V.) us — 7) Ы 


д); Itis easily seen that the null hypothesis in the 
Wing hypothesis in the y’s: 


and r, s = 1; 2; ы 


= ша = Kosi у j= 1, 2, 
Le, ma) = Ла). But this is the 
of the equality of mean ve 


——— 
a 
- —.———————————UÁÁÉÉStÉÉEN 

ET ти ЫЫЫ 


———— JoO——— 
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"NE д. ое с 
и n, +m (ja = Fo) W Gar — де) 

_ Tale кеі ды йе = RRS E. 

E EA > > wa — 9.) 0а - 9.2), 


where w™ is element rs in the inverse matrix ТР”. This statistic has the gu 
distribution with т, + nə — 2 degrees of freedom. 
To test the hypothesis at level а, enter 
Тп, +n: — p) 
(и + n. — 2) — 1) 
in the F table with p — 1 and n, + т — p degrees of freedom. If 


Т + n, — p) : T 

a tn Be oD > Ё„(р— 1,m + т p) 

reject the hypothesis; otherwise accept. 

The general case for g populations, 

is given by Heck [9]. The extension is obvious. 

of group means, one computes the between group 

products to obtain the elements of the matrix B, s 
this matrix is 


of which the above is a special case, 
From the g by p — 1 table 
s sums of squares and cross 
ay. Thus element rs of 


2 5. Ue 
b= Van = FIG = Я) = D dall. — nj. d... 1 
kel Le 


where r, s = 1,2, --:,(p — 1). For the error matrix W, compute similarly 


the sums of products, so that 


о nk 


о nk 
Wee = Ж py (yis = d.) — d.) = >, Dy кй,» — 
=1 ізі 


P» Tad exl. * 
k kal i k=l 
У: Фе» The various test statistics 
proposed are proportional to some function of the product matrix BW. 
In the literature on multivariate analysis, there have been three ap- 
proaches to the distribution problem. Wilks [17], starting with the likelihood 
ratio criterion, derived the test statistic | I + BW 1 ‚ which is obviously 
equal to the inverse of the product of the characteristic roots of a 2 BW ); 
I being the identity matrix. Hotelling [10] has proposed the distribution of 
tr BW or of the sum of the characteristic roots of BW . Roy [13] has 
proposed the consideration of the distribution of the maximum characteristic 


root y^ : : discussion of these three points of view consult 
otot ВИ, Pore MAT E robability tables available for 


Anderson ([1], pp. 221-224). There are пор Не ун 
the first two teat statistics although the exact cum! ulative distribution of the 
JS given by ап infinite series of Х 8, the first term of 


> -1 
In the above formula, ў. = ?% 


determinantal statis imation to the whol 
which, for any теаво Ме N, gives an excellent approximation to the whole 
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series and is quite easy to compute ([1], p. 208 and [12], p. 261). Several 
tables are available giving critical points of the distribution of the maximum 
characteristic root, the most extensive to date being due to Heck [9]. 

As an illustration of the Т? procedure, and also to compare the F tests 
with the T^ test, the two smallest groups, B and E, are selected for testing 
the quality of profile shapes. The mean profiles on the six items are given 


in Table 3. The analysis of variance performed on these two groups only is 
displayed in Table 5. 


TABLE 5 


Analysis of Variance 


lmum—————— I OIM 


Source d.f ss 


.ї. M.S. F 
ы NN NNNM 
Tests 5 733.03 

Groups 1 16.25 16.25 

Individuals within Groups 20 111.84 5.59 

Groups x Tests 5 105.56 21.11 Е. = 4.84 
Individuals x Tests 100 435.58 4.36 

within Groups 


Е c ба 2. 


To test the hypothesis of no group-test interaction, that is, of equality 
of the profile shapes, consider F, ; distributed as F' (5e, 1006). From the pooled 
sample variance-covariance matrix, an estimate for ¢ is .0727. "Therefore, 
for the approximate test, enter F, = 484 in the F distribution with 3 and 
67 degrees of freedom. This test yields 


.005 > РЕЗ, 67) > 4.84] > 001, 
with 


P[F(3, 67) > 4.84] zz .004. 
The minimum value є can assume is 1/5, Ther i 
v 4 А efore, for the conservative 
test, enter P, in the F distribution with 1 i 
я and 20 degrees of freedom, This 


05 > P[F(1, 20) > 4.84] > .025, 
with 
P[F(1, 20) > 4.84] zu 04. 

Аз previously indicated, to carry out the exact test it is necessary to 
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bins the dimensionality of the vector from p to p — 1. This is accomplished 
Ы subtracting az ‚хз , 4 , 25, and хо from 2, , obtaining a 5-dimensional 
vector y on each of the 22 individuals. The mean vector for group B is 


й& = (4.077, .692, 3.923, 5.692, .538) 
and for group E is 
2% = (7.556, 4.444, 3.889, 6.222, - 444). 
The pooled sample variance-covariance matrix in the y’s is 
5.057 2.854 .382 .560 2.384 
7.250 .906 3.994 2.196 
ТР = E d 5.090 5.696 1.155 
12.716 3.002 
5.272. 


with n, + n, — 2 = 20 degrees of freedom. Therefore, 


fits” fa Woe 9 
a D ie (jz — Gs) (jz — Ўв) 


27.048. 


Т? 


M 


Now enter 
TO т фт p. 2508.15. 4.328 
т --7.—2 Da 1 20 5 


(p — 6, the number of z variables) in the F distribution with p-1=5 
and n, + ne — р = 16 degrees of freedom. This exact test yields 

025 > PIFG, 16) > 4.328] > .01, 
With 
£g .01. 
в implied than that all three 
s not appropriate to compare 
e tests with the exact test based on the 
s of the latent roots previously discussed, 
T? in the case of two groups. The reason 


involved; one based on а function of the 
of the ratio of linear combin- 


Р[Е(5,16) > 4.328] 


sts, по more i 


By presenting these three te К 
e .05 level. Iti 


tests yield significant results at th 
the approximate and conservativ 
distribution of any of the function 
all of which reduce to Hotelling’s 
nm there are two exact tests ere 

ч s 
ations ee | rre on toe enphosined that the first two tests given 
In the example are approximate and conservative for the latter. The conserva- 


110 PSYCHOMETRIKA 


tive test provides а procedure which is more than "rough and ready" and 
yet saves considerable time since 16 does not require a matrix inversion nor 
even the computation of a covariance matrix. This is particularly true when 
p, the number of variables, is large and the number of samples is greater 
than two. 

The question of electronie computers is another matter. Given the 
availability of a classical analysis of variance program and the availability 
of à combined program to carry out the multivariate analysis of variance 
involving the between samples variance-covariance matrix, the inverse 
of the error variance-covariance matrix, and the extraction of the maximum 
latent root of the product of the two matrices, it is very likely that the former 
would require less machine time. However, the difference is probably of no 
practical importance and the exact procedure should be used. 

A more fundamental question relates to a comparison of the two exact 
tests involved. Are the multivariate analysis of variance procedures depending 
upon the distribution of BW"! more powerful against all alternatives than 
the distribution of the ratio of linear sums of x” variates? It is not clear that 
this is so, particularly with regard to. the analysis of profile shapes where 
the former procedures must reduce the dimensionality of the random vector. 

If one does decide to use the F tests in an analysis, the following series 
of steps are suggested. After finding the traditional analysis of variance 
table, first test the appropriate observed F value in the F distribution with 
full, i.e., unreduced, degrees of freedom. For F; , for example, this would 
be F with (p — 1)(g — 1) and (p — 1)(N — g) degrees of freedom. If Fs 
is smaller than the o critical point, one can stop here, for the null hypothesis 
will not be rejected with further manipulation of degrees of freedom. If 
the observed F is significant, then one proceeds to the conservative test 
where the degrees of freedom are reduced by a factor equal to 1/(p — 1). 
For F, , the appropriate F distribution is F (g — 1, N — g). If this test leads 
) cance at the « level, one can at this point reject the null hypothesis 
without further testing. However if the conservative test is not significant 
then it 15 suggested that the e be estimated from the variance-covariance 
matrix and the approximate test; be carried out. 

Number of Individuals Less than the №. 

As indicated in the introduc 
or in the case of g groups, if (№ 
variate procedures. The reason 


umber of Variables 
tion, in the case of one 
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Unequal Variance-Covariance Matrices 


in A Б the most important uses of the conservative test is 
талын. 1 қ lere ane cannot assume the equality of the unknown 
шр, Ил и matriees in the p-variate normal populations being 
noted zu ERE E case, there are no exact procedures available. It will be 

at this ease, p = 1 and g = 2, reduces to the Fisher-Behrens problem. 


RS pei in order for the F statisties to be unbiased, it is necessary to work 
я 1 onl sample sizes in the groups, ie," = °°" = т, = n. 'Тһегеїоге, 
"ев Zn, ks gn. Ц can again be shown that the respective numerator and 

ominator quadratic forms entering into 7, , F2 , and Fs are independent and 


have t 5 Soe <. 
ave the same expectations. Now, however, when an F distribution is used 


to approximate these F statistics (see [5], theorem 6.1), it turns out that 


p are different factors reducing the numerator and denominator degrees 
reedom, and these in turn differ for the three F statistics. Here again it 
сап be shown that these es have lower limits which when applied to the 
appropriate degrees of freedom result in à conservative test for assessing the 
Significance of F, , Fẹ , and Fs by entering these in the F distribution with 1 
and m — 1 degrees of freedom. Tt is of interest that the P, test, when p = 1 


еф g = 2, is а conservative test for the various approximate solutions given 
о the Fisher-Behrens problem © wo means with 


f testing the equality of t 
unequal variances (e.g., [15], p. 295). 
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DOMAIN SAMPLING FORMULATION OF CLUSTER AND 
FACTOR ANALYSIS 


Вовевт C. TRYON 
UNIVERSITY OF CALIFORNIA, BERKELEY 


les permit formulation of a general method 
luster and factor analysis methods are special 
cases stemming from decisions made at different stages of the general method, 
especially in defining an independent dimension. Key cluster analyses define 
i ariables drawn from the full n set. Centroid, 

define it by the n variables 


principal axes, and maximum likelihood analyses 
ifactor and second-order analy- 


(raw or residual, weighted or unweighted); bi п у 
sis, by both types of selection; square root analysis, by one variable. Key 
cluster methods can be designed to test hypotheses. 


The definition of a variable as the sum of a sample set of scored responses 
(e.g., to test items) selected to be representative of a defined domain of 
behavior is a basic principle of psy chometries. This standard practice may 
be expressed in a simple algebraic fashion which leads to an integration of 
the plethora of formulations of the reliability coefficient [39]. When a test is 
included among variables, domain sampling algebra also provides à definitive 
solution of its communality [40]. These principles have been shown to imple- 
ment the broad logic of multidimensional analysis by the psychometric 
procedures called cluster analysis [42]. The most generally applicable compu- 

the CC method, has also recently been 


tational variant of cluster analysis, 
published [41]. k қ А : 
Completing this group of papers on domain sampling formulations, this 
article has as its purpose, first, to state the general case of multidimensional 
analysis, and, second, to develop from it important special cases that are 
: pecial forms have, how- 


variant methods of cluster analysis. Some of these $ 


ever i over the last half century as factor analysis 
з bw айел ыс man [29, 30], Thomson [31], 


methods, their main originators being Spear қ 

Burt (2, 3], Kelley |19, 20], Hotelling 14, AU aia A p Seid 
/ thods of Spearman, епеу, , 

[13], and Lawley [21]. The factor тео ооа. 


А ic fa 
and Holzi Е ;ved as issuing from the basic 
olzinger are conceive results from un derlying, uncorrelated and 


assumptions are that a test scor r 
additive true (general and multiple), specific, and e а 
restrictive assumptions of factor theory are difficult to ae Зи Ее. 
bg antes! тані DA ir a mampi 
factor methods are recast as variants of clus а. гы. ү 


about the components of a test score 
113 


are unnec 
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TABLE 1 


DECISIONS THAT DISTINGUISH VARIOUS METHODS OF MULTIDIMENSIONAL ANALYSIS 


i PivoT |cEN'L 
KEY CLUSTER VARI- | AND | GENERAL CLUSTER 
REGIONS - JABE ей 
ОЕ [ 21|3|І4|5|6|71879 
Resid Cluster| Precluster Factor Analysis Variants 
DECISION 7 3 i 
Preclus|Ration Prin- | Maxi 
Tot | Cum Cum | Cum Баке Bi- cone cipal | mum 
ray en Com | Com (244 factor Um n| Axes | Like- 
(РСС)° | (RCC) 8" (Wat) |linood 


PRELIM DECISIONS 


а Reflection Yes Yes | Yes | Yes | Yes | Yes Yes No No 
Factor No No Yes No No 
b Precluster No No Empiric Ration] Yes No No No No 
Factor | No Yes | Wo No No 
DIAGONAL ENTRIES 
C Initia! Com- Simul d 
munalities | Quad Approx Approx! Approx Approx Арргох |Арргох |Approx| Мо 
Factor 


Unities| No 


DIMENSIONALITY 
d Defining Var- 
iables of C, 


Factor 


Highr \Unities| № 


n and 
x Sy Sy 5, |Зоп1 5 


х 
7 
+ | К and. Sy 


п (м9!) | п (wat) 
n(wgt) | n(wgt) 


e Partial Com- 


munalities | Sim $ | Sim x Sim Я | Sim x Sim Я | Sim | Sim x 
Factor 


Iter Iter 
a - АрргВ| Sim X | Iter Iter 
f Terminating d 
Criteriod | T T m | 5g T Resid r 
Factor Salrent Residr 
g Reiterate Г 
Factoring No Yes | Yes Yes Yes | Yes Yes Yes |Progk 
Factor No No No No |Progk 
STRUCTURE 
h Oblique 


Factor 
SCORING 
i Oblique Dimen- 
Sion Scores 
Factor 


Analysis |From d From d b,d 


Е tor analysi Ce = 
Ка formi oF the method’ d; multiple group"), see text 
“Specified for the special case о 
9 Sampling criteria may be used i 
"Regression in theory; 


From d|Rotate [Rotate |Rotate 
№ 


r Regress Regr 
2 |Regress' Regr? | Regr 


no initial communalities are necessary 

f s=] and communalities unity 

n place of or in addition to the T criterion 
cluster score by X in practice 
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" — plan of this treatment is represented schematically in Table 
"I genera, case of multidimensional analysis consists of progressive 
ачи. їп each of which decisions by the analyst are required. 
ot egions are summarily described in the lettered rows a to т, extreme 
ue umn of Table 1. The numbered columns 1 to 9 are the special cases, 
ach defined by the particular pattern of decisions listed down its column. 
The first four cases form the Key Cluster methods, columns 1 to 4. The 
тешр groups are other cluster analysis variants known also as factor 
analysis methods (or schools): their descriptive names are given below the 
column numbers. For example, column 7 defines the Thurstone centroid- 
simple strueture method. In this group of methods, note that in each region of 


decision (or row) there are two entries. The first entry is a decision resting 
on the conception of the method as а v: 


MASA ariant of cluster analysis. The second, 
in italics and labelled at the left as “Factor,” is the decision made in orthodox 


factor formulation. For example, down column 7 Thurstone’s decisions are 
represented by the pattern of second, italicized, entries. 

The general case of multidimensional analysis is given in the next section 

of the paper. Each region of analysis is taken up in order, the principles 

f the types of decisions listed 


involyed being illustrated by referring to some 0 
along its row. In later sections the special cases will be taken up successively, 


that is, for each column of Table 1 the nature and rationale of its pattern 
of final decisions will be given, first, conceiving the method as a variant of 
cluster analysis (first cell entries), and, second, as an orthodox factor formu- 
lation (second cell entries). 
General Case of M ultidimensional Analysis 

'The basic data of the analysis are the intercorrelations between scores 
on variables X,, Xs, 5322777? Xx; The over-all objective is 0 determine 
and measure the smallest number, k, of dimensions that reproduce the inter- 
correlations, entered as side elements in a correlation matrix. The successive 
Stages of the analysis, rows 4 to i of Table 1, achieve this objective. For 


convenience, these stages are grouped under five subordinate objectives, 
lettered A to E below. 


A. Preliminary Decisions (Table 1, rows a, 0) 


" Reflection of Variables (row а). 1 DM "Em 
Whether to reflect variables is the method of computin: 
(squared factor loadings), TOW € In the special cases of columns 1 to 1 , Where 
the simple summation (Sim х) f developed pe is used, 
reflection of variables denoted by “Yes” in row 4; js required. ~ А 
Preclustering V. ariables (row 0). Preclustering variables patoa dimeiaora 
analysis permits certain abbreviated or rational variants of тш ш 
analysis (columns 3, 4, 5). In the other variants test clusters are loca 


desideratum in deciding on 
artial communalities 
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during the dimensional analysis (1, 2, 6) or by rotation of dimensions (7; 
8,9). 


B. Determining Initial Communalities (Table 1, row c) 

The diagonal entries of the matrix are “ 
are selected as diagonal elements because th 
k, of necessary and sufficient dimensions, or uncorrelated cluster domain 
scores, that will reproduce the intercorrelations. If one used the reliability 
coefficient, т, , of each variable as its diagonal element, more dimensions 
than k would be required, and if one set each diagonal value at unity, even 
more dimensions would be necessary. 

As a correlation coefficient, the communality 
as the correlation between the observed variable X, and a hypothetical 


construct variable X,, measuring a different behavior property than X, but 


having correlations across the n — 1 other variables equal to those of X, 
(40), formula 1) ie., 


(1) № = т 


Аз а variance, the fundamental definition of h 
variance of the observed scores of v that is 
variable-domain score, C, ; defined as 


(2) GOES елы he des 
in which the observed e 
from an infinite set of с 


self-correlations.” Communalities 
eir use yields the smallest number, 


№: of a variable v is defined 


.... 


2 is the proportion of total 
predictable from the construct 


тә ) 


"Scores are defined as опе sample v. 


ariable drawn 
onstruct variables, all members of 


я д : which measure 
different behavior Properties, but whose correlations with then — 1 variables 
are proportional to those of о, From this construction it follows that ([40], 
formula 5) 
(3) № = ro, 
Propor tionality of the correlations of v with those of another vay iable 2 means 
Specifieally 
(4) Тыр = а, constant (іс ЕТІ N30 Æ i j) 
Taking the f 1 1 c i i 

ng the following as an index of Proportionality ([41], formula 6; [43], 

S rar) 

(5) Р = >т? Уу? Gj = ils 


È INU 1, 


he definition of 


the variable-domain 
a domain of 


variables with 
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Table 1, row c, note that in TC analysis (to be described later) a solution is 
attempted by a quadratic equation. But in remaining methods approximations 
are taken; after reiteration of the factoring procedure, row g, final converged 
values are achieved. 


C. Determining Dimensionality (Е actoring) (Table 1, rows d, e, f, 9) 
The object is to determine the value of k, the number of uncorrelated 


composite variables or independent dimensions, C1 , C2, *:* бу 7°73 Ge 
that could reproduce the correlation matrix, including diagonal communali- 


ties, 

(а) The communality, № , 
k partial communalities (squared factor 1 
(6) М ә М, FR ЕВ. 

(b) The correlation coefficient of v with each other variable 7 is repro- 
duced, i.e., the observed r,; equals 
(7) теа ыы ЖЖ п, Мы, 
ual correlations after removing the 


of any variable v may be partitioned into 
oadings) or A2, values as follows: 


or, said another way, each of its resid 
variance from dimensions 1 to k is 


(8) ints = a ты = 0. 
с. (Ап Orthogonal Factor) 


Definition of an Independent Dimension, Facto 
defined as the following in- 


(Table 1, row d). The score С, is à composite, 
dependent cluster domain score: 


(9) а. = to) s e ns ашығы 

The defining variables of the dimension àre selected observed variables a, b, 
* , s2 , taken from all n variables. The C's are their Pei een scores,- 

defined у ennt r means that scores on preceding imensions 

d by (2). The ргевопр r © dependence о О. 


C._; are held constant, thus establishing the in 


Же” i ) 
ariable may appear 1n different dimensions, 


) 


: than n. 
Чаа by a cluster of а, vanes ш the defining variables are 


axes, and maximum likelihood factor analysis, pere 
indiseriminately a general cluster of all а mete different 
differ from each other in the value gh Ж nsion Cy is а general 
component variables in (9). In bifactor analysis, Wo Өлү с. In pivot 
cluster of all п variables, but later dimensions ате ey 
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variable (“square root") analysis, each dimension is defined focally by one 
variable only. 

Recall that each C-variable is a variable-domain defined in (2) as a 
composite of z-scores of variables measuring different behaviors but showing 
proportional correlations, hence the use of communalities in the diagonals. 
Were the z-scores in (2) defined as different test samples of the same behavior 
domain, each C-variable would be the construct composite called the “true” 
or X, „-зсоге of the variable [39] and reliability coefficients would be diagonal 
entries of the matrix. Were each C-variable defined as the single z-score of 
the defining variable, then the diagonals would be unities. In multidimensional 
analysis, the definition leading to communalities in the diagonal is chosen 
for the reason given under objective B. 

The simplest weighting of the C-variables is to set all w’s to unity in 
(9), as in TC, CC, РСС, ВСС, centroid, diagonal, and bifactor analysis. 
This simple summation, as Burt calls it [3, 4], intrinsieally weights each 
defining variable of the dimension С, by its proportional contribution to the 
variance of С. , that is, by the sum of its communality and its correlations 
with the other defining variables. Another choice, characteristic of the 
principal axes and maximum likelihood methods, is the computationally 
arduous least squares solution of the sets of weights which yield the maximal 
sum of their Л: values, that is, of their partial communalities, 

Partial Communalities (Squared Factor Loadings) (Table 1, vow 6). 


The portion of the variance of a variable v predictable from a dimension С, 


is the square of its correlation with C, , called its partial communality, A2, , 


defined in (6). In the unweighted case, from the correlation of sums in the 
limit, 

10 м = m (Zr, š mn . Р 

UO E o tes XM +227, CTS ОГГЕ 

ку. кү ыш 18 simply the Square of the sum of the residual correlations 
ot v with the defining variables of C. . The denominator is simply the sum 
over the total submatrix of re: 


sidual correlations of these defini riables 
including the diagonal resi iti Э rae нна 
а н а] residual communalities, When ¿ = v, then ,h? is included 


A residual correlation, from (7) and (8), is 


(11) Jue теа = fy < (aay +. шн agi; 

and similarly for the тї; terms. A residual communality is, from (6) 

(12) Ме Lehi = № — Qi, Toce hey). | 
If one has chosen the definin 


5 variabl i i 
analysis as in POC and RCC anal Pot С, before the dimension 


ы ^ ysis, it is not necessar d. ће 
individual residual terms of (11) and (12). Only sums diede н 
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plus diagonals and sums of partial communalities on prior dimensions are 
necessary. Thus, from (11) the numerator of (10) is 


(18) т: Xni um dcc ШЕ а). 


@=а, = 48). 
When i = v, А? is included їп Zr,; . The denominator of (10), 


(14) х, + 2x45; = Era — (Eh)? pF (Shen), 
(i, j= а, +++ 18). 


Er,; is the sum over the raw matrix of s. variables including diagonals. 
This simple general formula (10), called “Sim >” in Table 1, row e, is 


used by all cluster and factor analysis methods, excepting principal axes and 


maximum likelihood (to be considered later). Recall that for different methods, 


it differs only in the value of s. ; €-« in centroid analysis, $. = 71. 
Terminating Criteria (Salient Dimension Analysis) (Table 1, row DE 
Аз a simple rational standard for terminating factoring, the writer proposes 
the communality exhaustion criterion. To end factoring by this criterion, one 
estimates the communalities of all the variables at the beginning of the 
analysis. Factoring then proceeds up to the dimension С» at which the com- 


munalities of the » variables are exhausted. 

Recall that the communality М is pasically defined in (3), quite in- 
dependent of the dimension analysis, as the variance of variable v predictable 
from its variable-domain C, . This magnitude is estimated before factoring is 
undertaken by computing formulas given later in the paper, (31) or (32). 
After factoring is under way, the variance of v predictable from dimensions 
Ci m ш om 5 Gis represented by Ai, , this magnitude being the sum of 
partial communalities of v up to and including hi, , as shown In (6). 

Writing the ratio of these two variances, 1.6.; 

lise ы 
(15) Е., = am 
that dimension С, at which the numerator 


faetoring may be terminated on 
when 


of (15) approaches the denominator, ie. 

hee 

1 р Se 1.000. 

16) Fi, = 12 
To evaluate F for each variable on each dimension " Wenn res 

would, however, be a complex procedure. Балаа Т. om approxi- 

for a given variable is subject to substantial error, both ap 1 өе of 

mation to the denominator term and in the initial po essaie sum (or 

the numerator. Less subject 10 these em ч ү i 

average) of the l-values over all n variables, funem 
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г zh? 
(17) Т. = zh 


Using Т. as a criterion, one stops factoring on the dimension C, for which 
Т, by (17) first equals or exceeds unity, i.e., where 


(18) T, = SE > 1.000. 

In practice, the T-criterion in (18) appears to yield the most salient 
dimensions [41]. Consider bias in T: initial estimates of communalities are 
usuall biased downward, suggesting that a salient terminal dimension 
might be rejected by T. This is unlikely because such a dimension would be 
the one for which Т first exceeds 1.000. The effect of an upward bias would 
be to accept nonsalient dimensions. To minimize such an effect, the analyst 
may set the criterion a little under unity, say, at .975. 

The T-criterion may reject later dimensions that would be accepted on 
sampling grounds. Many analysts may consider such rejection to be an 
unimportant loss, for such dimensions contribut 
and are usually difficult to interpret. 

Significant Dimension. Analy 
may wish, however, to accept al 
dimensions. The orthodox F- 
exhaustion indices represent 
but their sampling ch: 


e minor general variance 


ommunalities (Table 1, row 0). 
of partial communalities by (6) 


the difference between two successive it 
(6) becomes less than, say, a third of th 
value of its communality. Treating h, 
later, the approximate error is 


(19) шь, = (1 VN —G s 1 ауу. 
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The magnitude of №, at least 1, is usually trivial relative to N, hence leading 
to the final approximation shown. 


D. Determining the Structure of the Interrelationships (Table 1, row h) 


Having determined the dimensionality of the intercorrelations, one may 


relax the condition of independence and select or derive the k dimensions 
ter defined by the observed 


that may be oblique to each other and be bet 
variables. Key cluster analysis routinely locates those groups of variables 
which delineate the k most nearly independent oblique dimensions. Corre- 
sponding to the independent dimensions Сі, C2, ^^» С, there are the 
matched set of oblique dimensions, respectively, Cr > Сиз: Ск. Thus 
for independent dimension С. given in (9) there is ап oblique dimension 0, 


defined in simple summation form by the domain score 


(20) дуа Б a e 


Scores on the C-variables that form this composite have the same definition 
as in (9), but they are not residual scores 25 in (9). Geometrically, dimension 


C, is an oblique subcentroid in k-space. „мде 
If the analyst wishes to check on the clusterings indicated by the 
dimensional analysis with an eye, perhaps, to à possible reclustering of the 
he may employ 8 

Fig. 1). This model takes the k 


variables including those that had remained unclustered, 
geometric model (for an illustration, see [41], 115: Н ТИ 
dimensions as independent axes, and each variable as 8 point on them. The 
coordinate of each variable on any axis 0.18 its correlation, "scs (unaugmented 
factor loading), which by (10) is 


(21) Тұс» = he. . 
Тһе resulting interior model of variable-points ї8 pee te 3 
descriptive of the interrelationships among them аз the Вр Bee Ж, Ae T ia 
by plotting each variableelomain C, by its augmented correlation, this Беш, 
from the correlation of sums in the limit, 
(22) атты. 
4 i t distance 1.00 

The surface model, in whi Ш vari d uu 

el, in which all var З : ii 
from the origin, з {һе perceptual merit of revealing directly as 8 SU ace 
Separation of points the relationship between ndr OE 
апу one of the other variable-domains ©; +, eee the 
correlation" can, however, be computed directly irm re T 
communalities, i.e., from the correlation of sums m ' 


n таса rf hihi + 
Rotated Primary Factors). 


the most nearly indep 


The finally selected 


Simple Cluster Structure ( endent Б dimensions 


cluster domains of type (20) are 
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evident in the data. The degree of their interdependence is given by e 

intercorrelations. For any two such generally oblique cluster domains, ; уй Б 

С,;, their correlation is given, from the correlation of sums in the limit, as 
yis 


. Ут; 
(24) Тсс, = 


V Bra V 


e submatrix of the s; variables of Cy; including 
diagonal communalities, 2r;; is the same for the submatrix of 8; умы 
defining C,, , and 2r;; is the sum of the 5:5; Coefficients in their cross corre 
tion submatrix. Recall that, as stated under (9), a given variable may appear 
in more than one oblique dimension, a situation which would, of course, 
increase obliqueness. 

As an aid in interpreting an oblique dimension Cs: , one may compute 
the correlation of each known observed sample variable v with the dimension 
(rotated factor loading). By the correlation of sums in the limit it is 


where Zr;; is the sum of r’s in th 


Zr. 

25 "с, = =. 
(25) Cs; “Уз 
But of more interest theoreti 
general variation of which v 
domain C, in (2). This cor: 
simply, from {} 


(26) 


cally is the correlation of C,, with each kind of 
is taken as a test sample, namely, its variable- 


relation (augmented rotated factor loading) is 
16 correlation of sums in the limit, 


Testy, = Tc, №, . 
Difficult problems arise in simple cluster strueture analysis in those 


methods in which the defining variables of the independent C. dimensions 
are total clusters of all n variables, As sh 


8, 9, these dimensions must be rot 
nal or oblique clusters 


achieve rotation by objective 
analytic methods [5, 18, 24, 26, 28] Rotation, an unnecessary burden, is not 
required when the di key clusters, 


8 (Table 1, row i) 


| Ideally, the best estimate of an individual's sample score С,, on any 
dimension C, is the regression of C, on then variables, i.e., 
(27) б 


% 2% 


1 hat in most factor analyses the 
measuring persons on the reduced dimensions is rarely 
tackled, and a main ben 


ы ЧИ 
—————— — 
ee 
—— йы. ms 
А Ara c 
«G—" ЧА 
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When, however, a dimension is defined by a key cluster, а good estimate 
may be secured from the cluster score, namely, the simple sum of the z scores 
of the defining variables of C, , i.e., 

(28) Coa = ез», A 
In this composite each defining variable takes an intrinsic weight proportional 


tions with the remaining defining variables. 


to the sum (or mean) of its correla 
“у” may be used in all but 


In Table 1, row 2, the simple summation, labelled 


the general cluster methods (columns 7, 8, 9). 
Te The. cluster domain validity of the observed cluster score (28) is its corre- 
ation with the full domain score, C, in (20). By the sums formula in the 


limit it is 


Eh; + Zr 
sd Eri 


mjay ssij) 


(29) Тсасу = 


the sum over the submatrix of С, 


М " „б 
Note that the numerator in (29) is simply 
denominator is the same except 


including diagonal communalities, and the 
With unities in the diagonals. 
that fallibly measure the final 


А The relationships between cluster scores 
k oblique dimensions are given by their intercorrelations. Between any two 


Such scores, C,,, , Сз» , by the sums formula this correlation is 


(24) but with the unities in the diagonals. 


(30) Тсс, = SAME ав 


Multidimensional Analysis 


Special Cases of 
ality (ТС) and Cummulative Communality 


Key Cluster Analysis: Total Commun 

(CC) (Table 1, columns 1, 2) 
. The TC and CC methods directly apply the general formulations out- 
lined above. As shown in Table 1, columns 1 and 2, the correlation matrix 18 
Initially made positive (row a) by conventional reflection methods (see [10], 
Table 16.13) in order to guarantee that variables chosen to define a given 
dimension will show correlations of positive proportionality 1n (4). 

An electronic computer is required in TC analysis to solve for the 


Communalities by a simultaneous quadratic formula (row c). Difficulty has, 
ution in empirical matrices 


however, been experienced in achieving ® sol 
[16, 17. Тһе imd therefore been developed [41] to meet the 
Possible failure of solutions by the quadratic formula. CC analysis starts with 
approximations to communalities, i ionality analysis 1$ reiterated 
until convergence is secured. СС analysis is thus а procedure alternative to 
TC, being a method that provides 2 solution in all matrices, and one that may 
be programmed either for electronic or desk calculator computation. 
Solution of the communalities in TC analysis 18 based on the fact that 


124 PSYCHOMETRIKA 


the communality of any variable v is the squared multiple correlation between 
v and the remaining n — 1 variable-domains ([40], formula 11), 


(31) № = Ж буы бузы (i zv. 


Solving for the communalities of all variables requires a simultaneous solution 
by reiteration of the n quadratics of type (31). 

Various initial approximations to communalities required in the CC 
method are available [40, 45, 46]. On domain sampling grounds, the preferred 
estimate of л: is “Approximation B" ([40], formulas 29, 30), the Spearman 
formula, computed from a cluster of reference variables whose correlations 
are most nearly proportional to those of v. Here, 


(32) № = >т, т.т: @,] = %%<), 
where both 7 and j are the three variables showing the highest, P?, and P2; 
values, respectively, by (5). 

Іп both methods the defining variables of each dimension C, anchor on 
а selected pivot variable, о, , that appears to show relatively high and low 
correlations with other variables. Such а variable would usually center the 
cluster obliquely to clusters defining other dimensions, 'Го locate the pivot 
variable a measure of “pivotness” of each of the n variables is first computed, 


namely, the variance of its squared r’s, The pivot variable », then is that 
variable whose 


(33) var (т) is the maximum (Р = Dyes 


A quicker and probabl 
is to choose the one wi 


“nji £i). 

y less sensitive means of selecting the pivot variable 
th the highest residual communality as given in (12). 
ork; selecting v, from a correlation distribution table 


оу . Three such variables àt à minim 

i } - vi) йу 0, & then any additional variable 

t may also be included in the cluster if its P? | value is equal to or above .81 
— Pi, , that is, if its 


(84) ла Е В.) 5817 


Partial communalities, row e, are com: 
puted by th ral formula 
(10). In TC analysis the fac кг qu dad 


and the factoring process is reit 


erated until the communalities converge 
(rows f, д). In both TC and CC ar tes 


nalysis, the dimensional analysis locates the 


PM 
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k oblic dimensi 

ue row i 
Eus | i s i 10ns, 200 h, and scores of individuals on the dimensions. 
TO , follow the formulations as given earlier under the general method | 


Key Cluster Ал ў 
у чий ш. Preclustered Cummulative Communality (PCC) and 
Rational Cummulative Communality (RCC) (Table 1, columns 3, 4) 
„ы! 


Вес . 
Recall that in TC and CC analyses the cluster of variables selected to 


define i i 
u 5 Б 2 
dimension C, is chosen during the dimension analysis. One may 
"NC H . z 
empirically in PCC, rationally in 


21 fen choose them prior to factoring, 

аша T Preselection of clusters makes multidimensional analysis & 

bita : оа operation because the complete residual correlation 

а ртов а ‘ial to сс analysis are not required. РСС and RCC analyses 
cedurally identical after the analyst has clustered the variables (see 


Table 1, columns 3 and 4). 
Mer Teen re ise (for Ф recent illustration see [38], one empirically 
pea 54% ariables into k clusters, Cr ere Q,, -++ , Ок’. Each cluster 
mn. н аз tight” as possible, 1.е., 18 composed of variables whose 
е саган ме maximally proportional by (5). Some variables may remain 
inertie › but their number is kept аз small as possible. As an aid in select- 
Bu PE one may use а correlation distribution table. 

Анну Жа RGC analysis the rational groupings stem from the analyst’s 
E. m w hich he generated the n variables under study. An investigator 
вен у conceives the n variables to sample different, behavior domains or 

perties of the individuals, such as the facets of Guttman [11]. The s 


varia » : с 
riables that fall in each such theoretical subgroup are a rational cluster. 
ized in k’ such clusters, though a few 


= n variables will usually be organ: 
у remain as isolates. 

” == in сс analysis, one starts both PCC and RCC analyses by computing 

"d cmap to the communalities, preferably by formula (32). Thereafter 

сна vork is procedurally identical to the CC analysis excepting that only 
п residuals are necessary. The mean residual correlation of a variable 0 


= 
vith any cluster C, is, from (13), 
(35 

) ы = а) 


Ті : 
he mean residual communality of the 


(i = 4,0" RHD 


variables that compose C, is, from 


(36) = я =а /[5) 2-5 - 
The quickest means of choosing the pivotal defining cluster of any dimen- 
munalities as given 


ean residual com 
n computed by the simple sum- 
г of PCC and RCC 


thus defined by 8 


si к 
OM is to select the one with largest m 
mati ). The partial communalities are the t 
anal on formula (10). As shown in Table 1, the remainde 
yses is the same as in CC. The final Е dimensions are 


Selecti 
ection from the k’ original clusters. 
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A complication may arise if one should run out of clusters before di- 
mensionality has been completely determined. If one wishes precision on 
dimensionality he would compute the n X т residual correlation matrix and 
would perform the CC procedures on it and on any later residual matrices 
that are necessary. PCC and RCC analyses can be made as precise as CC 
analysis. To do so the analyst forms new estimates of the communalities by 
(6) and then, as in CC analysis, reiterates the factoring procedures until 
communalities converge. 

Abridged РСС and RCC (Multiple Group Factor Analysis or “Роот 
Man’s Cluster Analysis"). The analyst need only spend a few hours of work 
on a correlation matrix if he is satisfied with an approximate multidimensional 
analysis. After reflecting the variables, preclustering them, and approximating 
their communalities as in PCC analysis proper he can compute the correla- 
tions between the resulting hk’ cluster domains by (24). He may, if he wishes, 
even skip the step of approximating communalities, leave the diagonal vacant, 
and use mean r's instead of Zrs in (24). 

The result is a k’ X k' matrix with diagonal elements of unity. The 
correlation between each variable v and a given domain C, is then estimated 
by (25); if communalities are not used, he would use mean т” instead of Zr 
values in (25). These calculations result in an n X k’ matrix. From a study 
of these data he may make a reasonable estimate of the dimensionality, 
interpret the oblique dimensions, and compute cluster scores. If he wishes a 
more accurate estimate of the dimensionality and of the most oblique k 
clusters, he can factor the k’ X k’ matrix by the diagonal method of f. actoring 
(see “Pivot variable analysis" below). By this means he quickly locates the 
k most nearly independent cluster dimensions. 

Orthodox PCC Analysis (Group, Grouping, and Multiple Group Methods 
of Factoring). In 1939, the writer published dimensional analysis by the 
PCC method in approximately the form presented here ((37], Sec. 7); the 
quick abridged form was also given (Sec. 5, and Analyses 14, 15a for centroids). 
Five years later Holzinger [12] and then Thurstone (83, 35] presented the 
abridged form. Thurstone labels it the multiple group method of factoring. 
At the end of abridged analysis he rotates the i: oblique dimensions to orthogo- 
nal positions in order to compute residual correlations, and to see if further 
dimensions might be necessary. In his book, Thurstone [34] added the group 
and the grouping methods (see also [6], ch. 11). They differ from the 1939 
and current PCC methods in the procedure of grouping variables in clusters 
on grounds of absolute magnitudes of correlations rather than of proportion- 
ality of correlations, our Р? criterion. 

Pivot Variable (PV) Cluster Analysis (Diagonal or Sq 

(Table 1, column 5) 


In РУ analysis (see Table 1, column 5), 


uare Root Factor Analysis) 


each variable т among the n 
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ви "кс be selected as the central sample variable of an oblique cluster: 
сее re generally in (20), Cu: here consists of three variables only. 
Mie Weaken vo var iables are those which, after the matrix is reflexed, yield 
са values with 1. On domain sampling principles one may thus 
rr alize К = n preclustered domains, б,,, 00-2 Саз”, С,,. After 
г.) dene: арргохиш e to the communality of each variable from its 
сенде ee by (32), опе computes the correlations between the 7 
E by (25). Тһеп а dimensional analysis of then X n matrix of rc,,o,, 
hishe тўе with diagonal elements of unity, is performed. The cluster with 
м : 7 column sum defines the first dimension; locating the pivot cluster of 
: imensions requires only computing residual communalities and select- 
ing the one with highest value. W. ith 


euh an electronie computer, however, one 
Il instead compute residual matrices and more sensitively select the pivot 
ariable by (33). If the defining pivo 


C t cluster of any dimension С. is denoted 
v» » then the augmented partial communality of an oblique cluster ва 
Special case of (10), i.e., 


x Мы = аас П): 
he numerator of (37) calls only for the residual interdomain correlations 
h each of the remaining n — 1 clusters; only 


> the selected pivot cluster wit 

а тия n X 1 matrix of such residual correlations is necessary. Factoring is 
minated when the sum of the augmented partial communalities of all 

oe over all dimensions, that is, the numerator of the T-criterion, first 
ecomes equal to or greater than the denominator term, n. 

Having now located k pivot clusters by the dimensional analysis, the 
analy st assigns each of the remain ables to а final set of k 
oblique clusters. Each may be assigned to that pivot cluster which defines the 
dimension on which the variable in question has its highest partial 
communality. Tighter clusters тау, however, be groupe 


4 by criterion (34). 
An illustration of PV diagonal factoring procedures applied to an inter- 
98], Appendix D 


el matrix is given elsewhere ([ ). The potentialities of 
. analysis should be explored. It is а rapid means of estimating К, the 
dimensionality (rank) of the matrix, and hence it could precede maximum 
likelihood analysis where foreknowledge of the approximate value of k is 
desirable, ` 

of сы analysis would give precis 
[o communalities. But it may ? 
bons the first factoring to determin 
o ers, these clusters may be us , 
(|. "ae am the communality of each variable, v- Else 
Б 0), formula 44) that h; is the squared multip 

set of I; oblique cluster domains, j.e. 
(88) м = Ro crore * 


ng n — № vari 


ne started with correct values 
uch values, as follows. 
endent pivotal 
t of predictors 


е results if 0 
be employed to find 5 
e the k most nearly indep 
$ reference ве 
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From knowledge of the k oblique clusters by initial PV analysis, and "x 
initial trial values of the constants required in (38) computed from (32), 
(24), and (25), simultaneous solutions of all n communalities, reiterated in 
convergence, can be programmed electronically. Such a program may | е 
integrated with a periodic replication of PV analysis in order to discover 
whether the increased precision of the communalities produces changes in 
dimensionality. | 
Orthodox PV Analysis. One of the oldest factoring methods, diagonal 
factoring, used in PV analysis has recently been relabelled square root factor 
analysis [44]. In orthodox practice the analyst pivots a dimension on a single 
variable by arbitrarily inserting unities in the diagonal i 
unreflected r matrix. This practice prevents one from determining the di- 
mensionality of the matrix, rests the partial communalities rather unstably 
on the coefficients of the pivot variable alone, and leaves unclear the oblique 
cluster strueture of the variables. The method may, however, be useful in 


studies with large n as a preliminary analysis to locate the most promising 
predictors of a criterion. 


and factoring the 


General and. Key Cluster Analysis (Bifactor and 


Second-order Factor Analysis) 
(Table 1, column 6) 


Historically, the urge to diseover one 
behaviors provided the impetus to the ul 
dimensional analysis, It produced the two-fac 


and its subsequent generalization by Holzin 
analysis. 


general dimension in cognitive 
timate development of multi- 
tor theory of Spearman (29, 30], 
ger and Harman [13] to bifactor 


follow the same pattern 
definition of the dimens 
to the first dimension C. 


except for the one particular of the 
the deviation is only with respect 
n general formula (9) as the sum of 
= n". In the first residual correla- 
CC procedure is carried through, 


[13], the orthodox р 
not ables are preclustered into k’ clusters, 
each consisting of variables which (in 


drawn from each of the remaining k’ 


— 1 clusters. The first dimension, C; , 
is defined by this general cluster and is 
partial communalities of all n variab 
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g-saturations. In the first residual matrix and thereafter, the dimensions are 
successively defined in turn as each of the residual k’ — 1 clusters (group 
factors). The partial communalities are computed by approximation (32) 
but for the defining variables of each key cluster only. Zero partial communali- 
ties are by fiat assigned to variables in other clusters. This procedure is 
unwieldy computationally. Results from it will correspond closely to those 
more efficiently achieved by the CC method, modified as in Table 1, column 
6, by defining the first dimension as à general cluster domain, the remaining 
dimensions as key clusters; preclustering is unnecessary. 

Orthodox Second-order Factor Analysis. In Thurstone centroid analysis, 
when a generally positive matrix of correlations between primary factors 
results from rotation, the correlations between these first-order factors may 
then be subjected to a new centroid analysis on one dimension only. This 
dimension is termed a second-order factor. On domain sampling principles 
the k correlated primaries represent hypothetical oblique clusters which, 
unlike the oblique clusters discovered in key cluster analysis, are normally 
poorly defined by actual variables. The second-order factor is simply a com- 


posite general cluster domain—a battery score on the k oblique clusters. This 
general composite is difficult to interpret because of its vague omnibus 
character and because of the complex redundancy of some variables that are 


common to tivo or more primaries. 

A lema АВВ тя if desired, would be soared 2n а 15 
analysis designed аз described above, namely, by defining НЕ rs шо 
C, as а composite domain of all n. variables without тейи lancy, 1 | 
mensions as key clusters. To illustrate that such a first dimension CRAS. 
closely to the "Thurstone second-order dimension (адаа S А NS ex 
Writer compared the correlations between the 11 VAI E we 2d 
Thurstone second-order factor ((7], Table 5, 18-19 Ari i i as 
correlations with the Ci dimension defined as an ошо undan | d 
cluster domain. The P*-value of the paired columns of correlations was у 
in the second decimal place. г fac. 

астас ris can be extended, of course, to m Kum 
dimension and to matrices with positive and purs P на а 
the Srat prdor оаа УО pone ыс, d de in the last eaten 
under higher-order composites of oblique clusters discu: 
of this paper (see “Designed reanalyses ): 


^ ; ;q- Simple Structure, 
General Cluster Analysis и Т шшш, 8, 9) 


Likelihood Factor Analysis) ae y 1 
In the remaining group of methods епо Wwe | [m Es 
cessively as general battery residual scores on апт 
Weighted or weighted. 
Unweighted General Clus 


Principal Axes, Maximum 


iers (Centroid-Simple Structure Factor Analysis) 
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(Table 1, column 7). 1f the analyst wishes to define each dimension С. as the 
total unweighted residual domain score on the n variables, he sets 8, = n in 
(9) and all weights equal. The procedures of CC analysis are now called for 
down to structure analysis, as shown in Table 1, column 7. Аз pointed out 
earlier, the indiscriminate use of all п variables requires graphical or analytic 
rotational methods to describe the simple cluster structure. Following 
Thurstone, the simple-structure factorist who uses graphical methods usually 
does not place the oblique rotated dimensions through oblique clusters of 
variables but rather the rotated dimensions bound the variables. As a result, 
the calculation of individual’s scores on these oblique dimensions or factors 
requires the use of the complex regression equation given in (27). 

This complexity usually leads the simple-structure factorist, after 
having laboriously located the "underlying" primaries through rotational 
devices, to abandon efforts to estimate the scores of individuals on them. 
Thurstone recommends ([34), p. 515) that the complex regression score be 
replaced by computing a simple cluster score on the nearest oblique cluster— 
precisely the type of cluster domain that is directly located and measured 
by key cluster analysis. 

Orthodox unweighted general cluster analysis has been fully developed 
by Thurstone [32, 34] and his followers under the name “multiple factor 
analysis.” In Table 1, column 7, if one compares the orthodox procedure 
(2nd cell entries) originally formulated on the basic factor theorem with the 
procedures based on domain sampling (1st cell entries), they are seen to be 
identieal except for certain unrefined features of the orthodox method: the 
use of highest 7/8 as initial estimates of communalities, the terminating of 
factoring by а statistical test of residual 778, and the lack of reiteration of 
factoring to converged communalities. 

Weighted. General Clusters (Principal Axes or Components) (Table 1, 
column 8). One may wish to attach differential weights to scores on the n 
variable domains in (9) in order to maximize the sum of partial communalities 
on each successive dimension. The definition of the dimensions in this case is 
otherwise identical to that in the preceding unweighted case. Procedurally 
(see Table 1, column 8), no initial reflection of variables is required, but an 


electronice computer is essential to solve reiteratively by least squares for the 


values of the partial communalities, row 6. 

In the orthodox method of weighted general clusters as described by 
Pearson [25], Hotelling [14, 15], and Kelley [20], considerable confusion has 
been introduced by their use of unities (and sometimes reliabilities) in the 
diagonals. А rationale for unity diagonals would be equally applicable to 
any of the other methods described above and equally inappropriate, for 
communalities are called for, as stated earlier in this paper (see also [13], 
ch. 7). The only basic feature that distinguishes the weighting of general 
clusters from the preceding centroid analysis 18 the decision to apply weights 


b, 
| 
9 
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in the definition of each dimension. A main feature that distinguishes both 
approaches from the key cluster method is the decision to set 5, equal to a. 
Аз а consequence both general cluster methods require rotational procedures 
in order to describe cluster structure. 

Maximum Likelihood General Clusters (Table 1, column 9). As with the 
preceding two methods, Lawley’s maximum likelihood procedures (21, 22, 
23, 27] determine X: general cluster dimensions, s, = n in (9). But Lawley’s 
dimensions are significant in the sense that the final partial and total com- 
munalities reiteratively determined from trial values are model population 
values that would produce a correlation matrix from which there is maximum 
likelihood that the observed Statistical matrix represents sampling fluctua- 
tions. A chi square test of residual correlations is taken as evidence that the 
model cannot be rejected. The procedures are onerous even for an electronic 
calculator, since models with X progressively increased must be tested one 
at a time. Efficiency is greatly improved if good trial values of k and of the 
partial communalities can be found to initiate the reiterative procedures, 
This maximum likelihood approach has special appeal because of its capabili- 
ties of significance testing, but its gargantuan computation requirements and 
the need of rotation to a final cluster structure are serious limitations, 

To sum up the three general cluster approaches and offer a prospectus, 
all three general methods define dimensions as indiscriminate composite 
domain scores on all п variables. This definition is a common limitation 
because it leads to the uncertain procedures of rotation to simple structure. 
Paradoxically, the Thurstonian unweighted case, the most generally used 
method of factor analysis [6, 8, 10, 34], is least justifiable. A few books, 
notably British ones, do put the method in the right perspective [1; 3, 911. 
Centroid factoring is not as exact as the principal axes method in determining 
dimensionality. It lacks a test of significance, unlike the maximum likelihood 
method. For a general cluster solution the analyst would now use а modern 
computer programmed for the principal axes or maximum likelihood methods. 
As for the computational simplicity of the centroid method, the key cluster 
methods are simpler and have the additional merit of routinely describing 
oblique structure without the need of rotational devices. 

Designing principal axes and maximum likelihood solutions so as to 
describe oblique cluster strueture would remove their present inadequacies. 
Coupling key cluster analysis to them would, it would seem, turn the trick. 

Preliminary pivot variable (PV) analysis would quickly demareate the 
esired set of Is oblique clusters. If a method of least squares fit of № axes 
through these oblique subcentroids can be devised, the result would be a 
Weighted key cluster or principal cluster solution. To increase the efficiency 
01 а maximum likelihood method a preliminary РУ analysis should, it would 
Seem, provide 5004 initial trial values, not only of dimensionality e a 
9t partial communalities (appropriately unaugmented). If this method са 
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be further modified to pass hypothetical Population dimensions through 
the k key clusters, rotation would not be required, but it would retain the sig- 
nificance testing features of Lawley. The result would be a maximum likeli- 
hood key cluster Solution, 


Rationally Designed Dimensional Analysis 


Designed TC and CC Analysis. An analyst may generate the h 


ypothesis, 
for example, that clusters among n, variables of type A (say, 


k he hypothesis, per- 
n sk i 
eng deem £n should he higher 


decreasing salience. After the T-criterion 13 met, then 


— g 
< 
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For example, will a small number of higher-order, or composite, clusters 
maximally predict the communality variances of the n variables? To illus- 
trate, a study of the oblique structure of sociological and social attitude 
clusters may suggest that one composite of sociological clusters and one 
composite of attitude clusters may leave but а minor amount of the com- 
munalities of the variables unpredictable. The test in this case would be 
а new dimensional analysis in which the first two dimensions would be defined 
respectively by the two composite clusters. The residual communalities of 
the variables would constitute the parts unpredictable from these two 
dimensions. Thurstone’s single second-order factor analysis is a special 
design, discussed earlier, in which one such general composite cluster may 
be so tested. 

Another type of reanalysis consists of bringing together in one common 
new dimensional analysis clusters found to be the most nearly independent 
sets in different prior analyses. To illustrate, using urban neighborhoods as 
objects, the writer performed three separate CC analyses: on sociological 
variables in 1940, on the same characteristics in 1950, and on voting variables 
in 1954. Each of the separate analyses yielded three salient oblique dimensions. 
The nine oblique clusters were then projected into a single common di- 
mensional analysis. This master analysis still yielded three salient oblique 
dimensions, demonstrating thereby the common tridimensionality of these 
characteristics over more than а decade. In such reanalyses the analyst may 
also wish to include brand new variables that he considers theoretically to 


belong to the common structure. 


Summary 
The general method of multidimensional analysis, designed on domain 
sampling principles, covers as special cases all the main types of cluster and 
vary primarily in special decisions 


factor analysis. The different methods cial с 
about the nature of an independent dimension. Such a dimension is defined 


in general as а composite score on & cluster of variable-domains. In cluster 
analysis terms the special methods of key cluster analysis, denoted as TC, 
СС, РСС, and RCC, define eac lected set or subgroup 


h dimension as а ве 
from all the т variables. CC analysis is the most generally applicable. Square 
root or diagonal factor analysis, called PV analysis, 


pivots each dimension 
on one central variable. Bifactor and s 


econd-order factor analysis defines the 
first dimension as a general cluster of all n variables, the later dimensions as 
uster metho 


key clusters. Among the general cl ds that define each dimension 
as а composite of all т variables, centroid factor analysis defines 1% аз an 
unweighted total cluster domain, principal axes as а weighted one, and 
maximum likelihood factor analysis also 8 weighted one but having the 

ting the number of 


iti i 1 tima 
additional feature of supplying 8 technique for es 
statistically significant dimensions required. The key cluster methods de- 
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termine simple cluster structure as a routine aspect of the factoring process, 
Whereas the general cluster methods require laborious rotations to determine 
structure. The key cluster methods can be applied blind, but they can also 
be designed to test hypotheses. 
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LEAST SQUARES ESTIMATION IN FINITE 
MARKOV PROCESSES 


ALBERT MADANSKY 


RAND CORPORATION 


The usual least squares estimate of the transitional probability matrix 
of а finite Markov process is given for the case in which, for each point in 
time, only the proportions of the sample in each state are known. The pur- 
pose of this paper is to give another estimate of this matrix and to investigate 
the properties of this estimate. It is shown that this estimate is consistent 
and asymptotically more efficient than the previously considered estimate in 


a sense defined in this paper. 
Notation 
The matrix conventions and terminology used in [2, 4, and 6] will be 
followed. Let m4 @ = 1, 2, 77°, GE = 1,2, , n) be the proportion 
observed on trial № in alternative category ? of a multinomial population 
based on a sample of size S. Let E(ma) = на)» Where 0 X в, € 1 and 
25 шь = 1 for each k. Let 


My cot usa 
М = 
Ma cot Us 


t t;; be the transitional probability that an observation which is in 
ve category j at а given trial be in alternative category ? at the next 


. , a). Define T; = Ша +++ te] and 


Also, le 
alternati 
trial (i, j = 12, 


fa 5% ы 


Let N; = [ma *** та), and let N be the a X (n — 1) matrix with №, as 
row 7. Also let 
Ші US № 


jou. "3 est 
and », = [ш с ша) and let v be the а X (n — 1) matrix with v, as row t. 
137 
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Then Е(М) = в, E(N:) = vi, and E(N) = >. Also, by the Markovian 
assumption, и; = Tp, so that v = Ти and Т = vp’ (uu)? if up’ is nonsingular. 

One should note that since the Markov process has a(a — 1) parameters 
all these parameters cannot be estimated unless n(a — 1) > ala — 1), ie. 
unless the number of independent т;, is greater than or equal to the number 
of parameters to be estimated. It therefore should be assumed that » > а. 
It shall, however, be assumed that up! is nonsingular and hence, the necessary 
condition that pu’ be nonsingular, that n — 1 > a. Other assumptions will 
be introduced as they become necessary. 


Estimation of Т 


Since M is not a matrix of constants, the usual proof in the theory of 
least squares (cf. [5], p. 55) does not apply in this сазе to show that 


T, = ММММ” 


is the least squares estimate of T; . One ean work conditionally on M and 
find the Т; which minimizes 


(Т.М — NJ(T;M — N)' = C,Ct, 


given M. In that case 
T, = ММММ”) 


if MM’ is nonsingular, as is shown in [2], and the estimate of Т, conditional 
on М, which minimizes С.С; for each 7 is 


T = NM'(MM?)*. 


Even if M were а matrix of constants (or even if one works conditionally 
on М), T; would still not be the “least squares" estimate of T; . Although 
the elements of C; have mean zero (this can be seen as а corollary of (2.14) 
of [1]), they are not uncorrelated and do not all have the same variance 
o’.* In a situation such as this, the least squares estimate of Т; is obtained 
by transforming С; to 


б, = C, A; ^, 


whose elements are uncorrelated and have the same variance, c^ = 1, and 
by minimizing 


6:0: = CAC! , 


* am indebted to a referee for pointing out that equal varia i 
to use the method of least squares for аот, but only лаг ps г! үа 
properties of least squares estimates, in particular having minimum variance (as defined 
e.g., in [3]) among linear unbiased estimates, to be realized. The referee also points out 
that cov (mi, Са) = 0, i.e., the error is uncorrelated with the independent variables in 
the relation mia = У; tij mix — Cu. This can be seen as a direct application of Lemma 1. 


7 
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where A, is the covariance matrix of С; . Let us first determine A; and Аг. 
Use is made of the following lemma. 


LEMMA 1. Cov (Min , тыла) = в, нь (P — ш.)/8, where t£? is element 
(i, r) of Т", 


Pnoor. Define 


Y. 41 if individual s is in category ? at time j, 
id 0 otherwise. 


'Then 
5 
COV (ma, , т.а) = 22 соу (Yom › ӘУ! 
sel 
[Pr (P5 =, Mace > Lh рай, 5/8 


= [Pr (Eg, 
= (Ри, nek — Hinkle n-s)/S- QED. 


1| Ernane = Hinai — Badirs-kl/5 


І 


s I af € ef 5 
When k = 0, t{ is defined to be б;, = E otherwise? the usual multi- 


nomial variances and covariances result, namely 


cov (Min y Men) = — ый S 


апа 
var Min = ua(l — Hin)/ S. 


Let Са be the kth element of C; к = 1, «~ ут — 1. Then 
S var Cu = S var (= ть — таға) 
ізі 


nm = Шз) F 2, (1 EX Hir) 


= Us Шығыны” 2 >; Lini = шак) 
{= 


ini’ 


ë a 2 
2 — mM 
йа + mm 7 > баш (> tano) 


І 


= шаны — У бшк, 
j=l 


and when k > l 


, 
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S cov (С › C) = Baa? ка ш.а) cm b» tnit O x ш.м) 
ізі 
EC > ашан (СУ = Mix) 
Fr 


sus > ti; >; Liu E рп) 


а 
(й+1-1) (Е 
wd > ТІГІЛДІ "шалайын ы 
"m 


= 2 > шша > [T <р шаа СУ - 0, 
r= j- 


since 


a 
(&-1) — Ик+1-1) 
tte = di . 


i=l 


Hence А; is a diagonal matrix with diagonal elements 


а 
Шаа — >= Ж . 


1-1 
Now Л; is an unknown matrix, but each и is consistently estimated by 
its corresponding m. Also, since 
plim № =v and ріш М = y, 


5-е 5-5 


it follows that 


T = (ш) = T, 


plim 
5-2 
во that elements of T are consistent estimates of corresponding elements of 
Т. Therefore, A; , а consistent estimate of A; , can be formed by substituting 
the m’s and the elements of 7 for the corresponding y’s and !'s in the above 
equations for the elements of A; . 
Consider the estimate 


T, = NA, M' (MA, M)" 


Gf МАРМ” is nonsingular) which would be the least squares estimate of 
T, if Ay! = Aj! and if A; were not a function of the Ёз. It is easy to see that 
Т. is also a consistent estimate of T'; . 

If R and Q are symmetric matrices then R > Q if and only if 
a(R — Q)z' > 0 for all vectors 2. Let us adapt the definition of minimum 
variance linear unbiased estimate of a vector-valued parameter given in 


([3], section 2.5) as follows. It may be said that of two consistent estimators, 
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say и and v, of a vector-valued parameter 0, u is asympotically minimum 
variance if the asymptotic covariance matrix of \/5 (и — 0) is less than 
or equal to the asymptotic covariance matrix of ^/5 (v — 6) in the sense 
defined above. | 


THEOREM. Let 5; be the asymptotic covariance matrix of VS (7; - Т) 
and let S; be the asymptotic covariance matriz V/S (T: — T;). Then 3; € 3; . 


Pnoor. We know that 
Me = ua + оа/м/8), 
since for fixed 1, ты is the maximum likelihood estimate of иь . Therefore 


{н = tu + 00/V/S) 


and also 


5; = tis + 00/ У) 


for all тапа j. Then write 
№ =, +X/ V5, М-а-Ү/У8, ша Âp = AF + 2/8. 
To terms of order 1/ VS, 
MM? = uu! + (uY' + Үш)/ VS, 
so that 
(MM)? = И — (up!) UY" + Yu) V Ви)" 
to terms of order 1/-VS. Similarly, 
(МАА)! = [I — (аА) (uAi' Y' + uZu' + АҒЫ) (ал ыу 
to terms of order 1/ У Я. Therefore, asymptotically, 
VS (T, Т) = [Y + Ха) — Тау” + Yu’) op’) 
and 
М8 (P, — T.) = (е + ХА + АГ Y) 
— ТАУ" + би + УЛ) и)". 
Since v; = Тұр, the above equations reduce to 
8 (T; — T) = (X — T. Y)u Qu!) " 
and 
VS (P, — T) = (X — РТА и)". 


Л ү 7 is A; hat 
But the covariance matrix of X — T;Y is A; , 50 t 
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5, = (и(или) e) 
апа 
Š, = (WAT н”). 
By using the Schwarz inequality for matrices given in ([3], section 2.5), it 
follows immediately that 


4 
4t 


he 


b 
t 


Thus T, is asymptotically more efficient, in the sense defined above, 
than is T; . In fact, a modification of the above proof will show that 7'; 15 
asymptotically at least as efficient as any other estimator of the form 
N,PM'(MPM?)", where P is a positive definite matrix. 

One should note that 7 is obtained by first computing 7 and then 
modifying it. One can just as well, via the same procedure as above, modify 
Т to obtain another estimate of T, T*, say. However, the asymptotic ef- 
ficiency of T* will be the same as that of Ў, for each 7, since T* will be of the 
form N;PM'(MPM")", where P is Aj! based on 7, i.e., 


Р = А + 0/vS5). 

It has been shown in [2] that 7 has the desirable property that eT = с, 
where e is the a-dimensional vector (1 1 --- 1). That this is not true of T' 
can be seen from the example of the next section. 

Discussion 


Let us consider Miller's example [cf. 6]. In his case a = 2 


2, п = 20, and 
T= p M 
08 861] 


Using this estimate and N and M, the diagonal elements of А, are found to be 
[.20 —.04 —.03 .23 .09 .09 —.01 .16 19 .02 

А —.01 .16 19 .22 15 —.05 .09 19 .12] 
and the diagonal elements of А; are 
[.11 .28 .25 .05 .12 .12 .22 .08 .02 .16 


22.08.02 —.04 — 91 1912.02.06]. 
Note that some of these elements are negative, How 
elements are estimates of variance (which are non-negative quantities) 


these "estimates" were replaced by .001 (zero cannot be u i А 
А А 5 sed в , must 
be nonsingular). It is then found that US A 


is 81 ^ 
—.02 49 


ever, since these 


дім 
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This example illustrates that e! > cin general. It also shows that elements of 
T may be inadmissible in that they can turn out io be negative. However, 
as Goodman points out [2], 7 also has this latter fault. 


We suggest in this case the estimate 


ћ = ім | 
10 .75 
To compare T with T, it suffices to compare the covariance matrices of 


T, = T, and T, (since T» = e — T, and T, = e — ТІ). An estimate of Xi 


is (M AT! M^)^!, and $, is estimated by 
(MMY (MÂ, MMM’. 


using 7 (and finding that all the diagonal 


After re-estimating А: by 
we used this estimate and found that 


elements of this estimate are positive), 


= | 1049 aH 
est Z; = 


—.123 .517 
апа 
z .059 —.149 
est 21 = | | 
—.149 .616. 


The easiest way to see that est 2, > est 3, , where > is аз defined above, 
is to note that all principal minors of est 5, — est X, are non-negative. 
Goodman also states in [2] that if the observed transitional proportions 


are available, they would clearly be more appropriate in the estimation of 
transitional probabilities. In [1] Anderson and Goodman give estimates of 
T and the asymptotic variances and covariances of the elements of their 
estimate when the observed transitional proportions are available. Calling 
their estimates ің, the asymptotic variances and covariances may be written 


as 
var £j; = м1 = &)/Ф: › cov (6, ti) = — stite /bi » 


1 i-k 
б = 
0 іжһ 


қ 
СЕЗІН ша 2 һа: 


ігі k=2 


апа 50, 


where 


and 


in the example under consideration, 


We can estimate ша by Mir 
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we can estimate Ñ, , the asymptotic covariance matrix of the estimate of T, 
(and hence Т, = e — Т,) when the observed transitional proportions are 
available, in order to see the improvement when this information is available. 
It is found that est ф, = 15.3 and est ф = 3.7 so that 


тый 5 E 2 
.000 .051 


using T to estimate T. Hence est $, < est $, in this case. 

А With regard to the amount of extra computation involved in computing 
Т, one can see from the form of the estimate that once 7 is given, the modi- 
fication process will take about as much time as the computation of T itself. 
Since we have no idea of the relative decrease in the variances of the esti- 
mates, we cannot discuss the trade-off between the doubled computation 
time and the reduction in variance. This trade-off is, however, an important 
practical factor in determining whether 7 or T is used. 
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AN AUGMENTED MODEL FOR SPONTANEOUS REGRESSION 
AND RECOVERY* 


Davi» McCoxNELLT 
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The Estes model for spontaneous recovery and regression is modified by 
relaxing the assumption that within-experiment stimulus fluctuation can be 
treated as negligible in the general case. The augmented model allows descrip- 
tion of stimulus fluctuation both between and within experimental periods. 
Application of the model to mock experimental situations illustrates some 


of its properties. 


d upon experimental findings reported in [5], and 
represents an effort to broaden the scope of Estes’ model for spontaneous 
regression and recovery [2] so that it may hold for a larger class of data. In 
the original model, Estes partitions the total population S* of stimulus 
elements available in the experimental situation into two sets S and S'. 
S is the set available during an experimental period, and S' is the set un- 
available during an experimental period. The numbers of elements in the sets 
are related by the expression N + N' = N*. The probability j, defined on S, 
is the probability that an element selected at random will leave S and enter 

' during an instant At of а rest period between experimental periods. Simi- 
larly, j is the probability that an element from S’ will enter S during that 


instant. 
These p 

In the augm 

perimental periods. As in the 


This paper is base 


robabilities are undefined by Estes during experimental periods. 
ented model they will be defined both within and between ex- 
Estes model, the equation 


. ML EXE ONCE 

NEN 1-7 

relates these probabilities to the respective subset sizes. The augmented 
model thus retains the basic notation of the Estes model. It will, however, 
be more convenient to regard S as the set in which direct conditioning or 
extinction of stimulus elements is possible, and s as the set in which no 
direct conditioning is possible. From period to period, and during rest periods 
between experimental periods, S* is considered to remain unchanged, ie, 
the same elements will be contained in S* and their number N* will be fixed. 
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The values of j and j' may vary from intraperiod to interperiod intervals, 
but will be assumed constant, on the average, within any one interval. 
Lastly, the proportions of conditioned elements in S*, S, or S' will be con- 
sidered to remain unaltered when values of 7 or j' shift. 

The general situation requires a stimulus-sampling operator which 
will carry out conditioning of new elements in S for any values of the param- 
eters j, j'. This operator, ¢, must be such that the proportion of conditioned 
elements at the conclusion of trial n + 1 is related to the proportion on the 
previous trial by an expression such as P,.; = ФР, . An operator invoked 
some time ago by Estes and Burke [4], such that Pası = P, + 0(1 — Pa), 
(where @ is the proportion of elements from S sampled by the organism on 
trial п + 1) will carry out conditioning in S when j and j' are both 0, but 
cannot apply unmodified when the exchange parameters are nonzero. It 
should be observed that the applications made by these investigators involve 
what Bush and Mosteller [1] call *experimenter-controlled" reinforcement 
situations, while extension of the operator to the present experimental 
context [5] entails “subject-controlled” reinforcement. This distinction, 
at least in the case of animal conditioning, may be considered that between 
respondent conditioning and operant conditioning. 

The operator ¢ will be applied in the subset S exclusively. Response 
probabilities in S’ and in S, respectively, immediately after a trial on which 
ф is applied will be related to probabilities immediately after the previous 
trial by the following equations. 


(17 Вии = РДІ = jy-FPRJNIN', 
апа 


0) Р, = P -—j-FPS'W'/N + в PA — j — РММ]. 


Equation (1" states that the proportion of conditioned elements in 
S' at the conclusion of trial n + 1 is equal to the proportion (1 — j') of those 
found in S’ at the conclusion of the preceding trial, plus the weighted pro- 
portion jN/N' of those found in S at the conclusion of the preceding trial 
(the latter having escaped from S into 87 during the intertrial interval). 
Equation (2^) states that the proportion of conditioned elements in S at the 
conclusion of trial m + 1 is equal to the proportion (1 — j) of those found in 
S at the conclusion of the preceding trial, plus the weighted proportion 
j'N'/N of those in S’ at the same time, plus the proportion 0 of the un- 
conditioned elements from the same sources (these having been sampled 
and conditioned on trial n + 1). Rewriting (17 and (2^, 


(1) па (1 = Ре [Ре =0, 
and 
(2) Poa = 0 = 9 = DP, -(1- УР) 6, 
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These equations are the basic finite difference equations for the augmented 
tion described by Estes and 


model; both the exchange-free learning equa 

Burke [4] and Estes’ general expression (equation (3) of [2]) for spontaneous 
recovery and regression between periods can be derived from them as special 
cases, For the first case, note that when j = ў = 0, (1) and (2) become 


(la) la — Pi = 0, 
and 
(2а) Р = 1 — ФР, = 6. 


These have thersolutions P7 =P), and Ps 1 — (1 — Po) (1 — 6% which 
are identieal with those of Estes and Burke [4]. 
0, but = 0, 


For the second case, note that when j, j are not equal to 


(1b) Pi. = (1 = РР. + ЎР, , 


n*l 
апа 
(2b) Р = 0 — Р. Р. 
From (2b), 
Р.а = (Е = Ра, 
Big = Pus == Pa , and Р; = ES xs ; 
J 
substituting back into (1b), 


М, POI - d 


Р, п+2 7 ( 
eristic equation 


This last equation has a charact 
J-j- УВЕ 
i 2 


whose roots are 


j-D.i1-i-il 
= , 


ті, 


Then, : 
бй ей М tem sedis 
= са" ag 


Thus 
Рн 


whence, on substituting in (2b); 
pie = 0/0 at T: 


From the equations for P, and Pi 


and Ps = (-//Йа ATUS 


Py = @ +o» 


Whence 
= = (Po в), 4 0а ма. 
СД 2 ы = -- 


а=; 
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'Thus, 
7 Li n 1 ГА 
Р. = = Po — Ра” + Po — =—— (Р, — PD, 
1+ 7( 09 2) + 0 j j C о 0) 
ог, ѕіпсе 
1 Ji 
ту =1— J, where J = ——; 
"ER : iq 
this can be written 
P, = РЈ + (1 — Ла] + Ра — Ла- а”). 
This expression is identical, when # is substituted for т, with Estes’ basic 
equation for spontaneous recovery and regression. The corresponding equa- 
tion for S’ is 
Pa = PW — а) + Poll — J(1 — a7]. 


For the general case, when j, j' and 0 are nonzero, note that (2) and (1) 
are in the forms, respectively, 


(8) Ша — Gu. — М, = 0, 
ала 
(4) фа — dv, — j'u, = 0, 


whereu = P,v = Pz-na-(1—0)0—j,b- (1—606jd21— at 
From (3), 


— №2 — ОШ. — 0, Ча — Qu, — 0 
ал Е раа j Uy = 


Upon substituting in (4), 


И. — аи. — 0 1 
(4а) = —“ — $n — au, — 0) — j'u, = 0, 
which becomes 
(4b) Чы — (a + ди. + (ad — j'Üu, = (1 — d)0, 
or 
(5) Pasa — (а + ФР, + (ad — ГОР, = ө. 


This is in the form 


(6) АР» + BP, + СР, = Dy, 
for which the characteristic equation is 


(7) y() = Ат + Br -- C — 0. 
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If the two roots of this equation are real and unequal, whether positive or 
negative, the explicit solution will have the following form. 


DW è 
8 керкү 1 2! 2. 
(8) Ра To + сит en 


It easily сап be proved that (7) can have no complex roots. It would have 


such roots when В? < 4C, or when, expanding, a^ + 244 + Ф < 4ad + 4j'b, 
or transposing, à? — 2ad + Ф < — 4j'b, or (a — dy < — 4%. But (a — d)? 
is always positive, as are j' and = (1 — 6)j, so the expression (a — dy « 
— 4j'b can never hold, and neither can the original expression BSAC: 
Thus 7, г must always be real. 


Now, 
ри 
9 да E 
(9) yo) ^ 
after substitution.* Thus, from (8), 
(10) pP,-1-ceonce. 


The roots of (7) are 

-B+ УВ? – 46 
(11) Е РЫ 
since А = 1. From this it can be shown 
0<т-<т”< 15 


(12) 
after substitution. Therefore P. = 1andP,—-1-da-te. From (2) and 
(10) 
P= ШЕ е да — 2(1 + ст + сл) — 0 
(13) p (1- 9j 
= 1 + + №, 
where 
(1-8 -—23) ‚ _ © — (1 — DU = D), 
(13a) № = „сс, 4 and № =~ С Je Bj 
Then PZ = 1, and Pj = 1 + kı + kz . From this, 
_ pa — 9) = 0 = Pod = 9i 
1-р) = a — 9 ж-да о | 
(10а) в = ( о) 1 am 
and 
ал —(1 — Po) — а. 
29 к Раоа 


В = (a +d, = ad — j' D 7 76 
1-7 


7% 
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Now, from (10) and (10а) it is evident that P, = QP (where Q is a variable 
positive coefficient) since с; is linearly related to Ро, and P, = Ос„(т»— n) = 
Ос» , because of (12). 

This relationship permits equal initial probabilities in the available 
set S to diverge as а period progresses, the higher curve obtaining for the 
higher starting probability in the unavailable set S'. It can also result in 
crossovers between initially disparate curves, such as those found in the 
relative frequency measures of the experiment [5] leading to this paper. 
Graphs of the augmented model, based on (10) and (13) above, and using 
dummy parameter values, appear in Figure 1. 

Figure 1 depicts the application of the augmented model to a mock 
experiment involving the same conditions reported in the empirical paper 
associated with this study [5]. Although care was taken to insure that the 
ordering of curves in Figure 1 corresponds to that obtained in the empirical 
study, this was done only to demonstrate that the results of that study 
could fall within the domain of the model presented here. No effort was 
made at curve fitting—until such time as new experimental predictions are 
made from the augmented model, curve fitting would be at best second 
guessing. 

The particular parameter values chosen do not in general uniquely 
determine the form of the functions shown; and these values were in some 
cases selected only to illustrate special points. The values of P, and P/ in 
period I were taken from the observed relative frequencies correct in the 
first minute of reinforced responding [5]. The high value of 0 (.8) chosen for 
periods I and II is intended to illustrate that even with very rapid acquisi- 
tion and extinction, reversals of initial probability relationships, such as 
that demonstrated in period II of the empirical study, can be expected 
between immediate and delayed extinction groups if intraperiod exchange 
between S and S' is permitted. The large value of j, together with the dis- 
proportionately small value of j' chosen, is intended to emphasize that the 
corresponding value of J within the two periods (.0141) may be markedly 
different from that prevailing between periods (J = .8 between periods I 
and II). 

In period ІП, one sees how a radical (80-fold) reduction in 0 affects 
terminal probability. Here, for the first time, response probability in 8” 
actually decreases across the period for two groups—a result not super- 
ficially obvious in considering the augmented model. 

The main object of the selection of parameter values for period IV is to 
illustrate the nonexchange learning situation for the same value of 0 used 
in the previous period, where exchange was permitted. Note that probabili- 
ties are stationary in the unavailable set, which in the augmented model is 
the exception rather than the rule. In the Estes model, this condition was 
assumed throughout experimental periods [2]. 
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Discrepancies Between Initial Probabilities in S and 85! 


Above it is shown how an excessively high proportion of conditioned 
elements in the set S’ can retard extinction and bring about a reversal in 
ordering of immediate and delayed groups in a simulated experiment. Now 
consider the case in which P4 < P, at the start of an acquisition period. 
Referring to (2) of the augmented model, and solving for AP, , gives 


(14) AP, = 61 — P,) + i1 — ВР, — Р,). 


It is clear from this expression that if the factor (P; — P,) is negative and 
sufficiently large, the increment AP, becomes negative. This implies that 
probability in the available set will decrease until such time as the condition 


" _ 61 — P.) 
| ЖАБ = LB 
holds, and AP, = 0. When 0 is large and conditioning proceeds rapidly, such 
an initial discrepancy may produce no visible effect on the conditioning curve. 
But when 0 is not large, there should be а detectable dip in response prob- 
ability in 8 at the start of an acquisition period. 
Figure 2 shows a low-6 dummy experiment, in which by the trial 20 
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Curves of proportions of conditioned elements in 5 and 5’ for а dummy experiment in 
which 0 = .01, į = .20, j' = .05, and the starting probabilities are Po =1,P = 0. 


the dip in available set probability has reversed. Note that after its initial 

intersection with the curve for the unavailable set, the available set curve 

increases monotonically below the other curve. These same equations may 

be applied to an extinction period by conceiving of the period аз one of 

acquisition of the previously unreinforced response, We may conjecture that 
° 
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when initial probability of the response to be extinguished is less in the available 
set than is the proportion of elements conditioned to it in S’, there should 
be detectable an initial rise in response probability, preceding the typically 
decreasing extinction function. In general, however, exaggerated discrepancies 
in probabilities such as could obtain at the end of an acquisition period 
during which values of j and ) are both low, will not result in these early 
phenomena in a subsequent period, because there is no reason to assume 
changes in the parameters j and 7. 

Without such changes the exchange of elements will proceed no more 
rapidly than in the preceding period, with the result that the arrival of 
elements from S" will be offset by conditioning at the same rate as in the 
previous period. This will be true also for the extinction phenomenon. Barring 
the assumption of changes in the drift parameters from one period to the next, 
the only way to demonstrate these phenomena is to produce the necessary 
probability discrepancies “artificially.” Consider, for example, certain 
elements introduced into the available set at the start of acquisition, con- 
ditioned to the reinforced response in sufficient number to raise the initial 
probability of the response in S markedly above the corresponding proportion 
of conditioned elements in S’. Then an influx of elements from S could be 
expected to bring about the initial dip in question. The elements introduced 
might be associated with transitory phenomena such as temperature changes, 
handling by the experimenter, or transfer effects from other situations. 


Period Spacing 

Estes, still with the assumptions that j = 7 = 0 within р Set E. 
terminal response probability for the reinforced response ш ig po Pe т, 
predicts that increasing time between periods will accelera’ eR Eun 
lation of conditioned elements іп S* [3]. The augmented що $ 5 h during 
same prediction, but qualifies it to the extent that if j and riod su 45 
ап experimental period, increasing the time E kis ualification, 
virtually no effect on response probability. To documen қ. Ші. са 
consider the extreme case of j = 7 = 1 within UE ind! (13) still apply. 
this condition, the general forms of the solutions (10) а 


— $. The only effect of the 
However, both equations now һауе roots F T Up Тен» 


: LGB rd 
) [ c 1 ё introduce a slight oscillation in 
hegative root, since trials are discrete, лн Е. elements in S and S". 


the growth curves of the proportions 25 rastically the 
But File жос of the dee inj and 7 ; poe SX. of Fw 
accumulation of conditioned elements m ж ent be indistinguishable 
trials the proportions of conditioned elemen d j' are large, so that equal 
from 1 in both 8 and 87. When, therefore, J 806 Sir in S and 8, it is 
limits are quickly approached by response oe t increase the over-all rate 
clear that increasing time between periods canno 


of conditioning in S*. 


Proportion of Conditioned Elements in S and S' 
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Estes also predicts that in a sequence of randomly alternated acqui- 
sition and extinction periods, response probability in S* will approach the 
asymptote z, the proportion of acquisition periods in the sequence, and that 
this asymptote will be independent of values of J and of the time between 
periods. In general, predictions for mixed sequences, whether based on the 
augmented model or on Estes' model, must be applied cautiously. Both 
models result in the prediction that т will be the expected value at time % 
following the mth in a sequence of such periods. But while the approach of 
the expectation to т is monotonic across the sequence, it is by no means true 
that for а fixed experimental sequence the interperiod asymptotes must 
each approach т. 

This point is illustrated in Fig. 3, representing four dummy experiments, 


Period 


I Rest I Rest Ш Rest IV Rest М Rest XI Rest VII Rest ҮШ Rest IK Rest 
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FIGURE 3 


Response probability in S and S’ for four dummy experiments involving 
of acquisition and extinction periods. 


a mixed sequence 
for each of which the proportion of conditioned elements in both available 
and unavailable sets is plotted period by period. Examination of the first 
and third os of these curves demonstrates that even with the Estes assump- 
tion (j = 7 = 0) regular oscillation can be expected to continue indefinitely 
for any аја ог for any group whose members are identically treated— 


24 
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in spite of the fact that the mean interperiod asymptote is clearly т. This 
oscillation should of course disappear if Ss are randomly assigned to different 
schedules and response curves are plotted as the mean of all measures at 
a given trial or time. The second and fourth pairs of curves in Figure 3 show 
what should be expected in the extreme case of the augmented model when 
j = j = 1, and when z is the same as in the first and third experiments, 
respectively. It is clear that the extrema of oscillations may be signifieantly 
greater than those for the Estes model; it is evident that this should be true 
to some degree even for j and j’ less than 1. However, it is equally clear that 
the mean interperiod asymptote in the extreme oscillatory cases is still z, 
as it is for the Estes model. The advantage of the augmented model for long, 
of periods would thus appear to accrue from its ability to 


mixed sequences 
interpret interperiod asymptotes which are inordinately higher or lower 


than the expected value of т. Since, however, some oscillation from period to 


period is to be expected with the Estes assumptions the intraperiod exchange 
The criteria of intraperiod 


notion should not be applied on this basis alone. 

reversals of order or nonmonotonic response functions should first be invoked, 

since only they are unique to the free-exchange situation. . 
The potential applicability of the augmented model to à variety of 

psychological situations appears to the author to be derived from the simple 

notion that stimulus fluctuation within even relatively short experimental 


periods can be evaluated by means of the simultaneous e 


without being relegated to the status of a variable "outside | t 
the mathematics of Estes 


is conceded that these equations do not simplify uth 
from digital computers. A 


basic model, but relief is in sight on this score Som of ations 
number of routines are now available for simultaneous solution of n equau 


i i f i ters such as the 
using trial va ants by use of high-speed compu t t 
нн Тый to be found in computing facility 


IBM 650 and 701 or 704. These routines are р à à 
libraries at Indiana and Ohio State, and are available by request. 


quations proposed, 
” the model. It 
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A MODEL FOR ORDERED METRIC SCALING ВУ 
COMPARISON OF INTERVALS* 


Бовевт Е. ЕлсотТ 


UNIVERSITY OF OREGON 


. This paper presents a model of individual choice behavior for appli- 
cation to experimental situations in which a subject is required to compare 
utility intervals (differences in subjective value). This model is contrasted 
with a weaker model, which is also derived. Both models generate ordered 
metrie scales, but differ in predictive power. Ап experiment on the utility of 
grades, which provides a test and comparison of the models, is presented. 


Coombs [2] introduced the term ordered metric to denote those scales 
which provide an ordering of the set of alternatives, or stimuli (the single 
property of an ordinal scale), and in addition at least a partial ordering of the 
distances between stimuli (utility intervals). Siegel [8] presented a method, 


based on a one-person game, which generated a so-called higher-ordered 
utility intervals. The present model 


metric scale, i.e., а complete ordering on 
le. The term ordered metric scale, 


also generates а higher-ordered metric sca r 
rays refer to a higher-ordered metric scale, as 


as used in this paper, will alw 
the term is used by Siegel [8], and will be abbreviated OM. 

Coombs presented no formal characterization of the OM scale, nor did 
Siegel; one of the purposes of this paper is to present such a characterization. 
Two important results will be presented in relation to the properties of such 
scales. First, it will be shown that it is possible to construct a scale which 
satisfies the above two properties (an ordering of the set of stimuli, and an 
ordering of utility intervals), but which is inadmissible as an ом scale. Тыз 
implies, of course, that the intuitive definition of an OM scale given above is 
inadequate, since it admits to the class of OM scales types which were not 


igi i it wi hown that it is possible to have two 
originally intended. Second, it will be 8 possible | ей 
or more models, all of which generate OM scales, but which differ in predictive 


power. 


The subject will be required to make judgments in relation to pairs, 


m me 
i imuli. F irs of stimuli, he will have the traditional 
triads, and tetrads of stimuli. For pairs | od eoo т 
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and tetrads of stimuli, he will be required to make comparative judgments 
of intervals. As an illustration, if three stimuli have been ordered by the 
method of pairs xyz, then the subject could be required to make the judgment 
whether y is more similar to x than to 2. The particular relation utilized (in 
this illustration more similar) will depend on the experimental situation, but 
in all cases the subject will be required to make a judgment which permits 
an inference as to the ordering of two utility intervals. Гог example, if as 
above the stimuli are ordered xyz, and y is judged more like x than like z, then 
it will be inferred that y is closer on the subject’s utility scale to a than to 2; 
ie., the utility interval between y and z is greater than the utility interval 
between x and y. If tetrads are used, the judgment will be of the form “the 
difference (along some specified dimension) between x and y is greater than 
the difference between 2 and ш.” 

Although the fundamental relation between pairs of stimuli is interpreted 
in this paper in terms of direct comparisons of utility intervals, as illustrated 
above, the formal structure of the model is applicable to any method, such as 
Siegel’s one-person game [8], for which each judgment implies an ordering 
of two utility intervals. 


Formal Structure of the Model 
'Тһе model is based on three primitive notions. The first is a set K, 
interpreted as the set of alternatives, or stimuli. The second is a quaternary 
relation Q on the cartesian product K X К. The relation Q is interpreted 
such that zyQzw holds when the difference in subjective value (utility) between 
ж and y is judged greater than the difference between z апа w. If the alterna- 
tives are objects among which the subject is stating preferences, then vyQzw 
implies that the difference in strength of preference (value) between x and y 
is greater than the difference between z and w; the subject more strongly 
prefers x to y than 2 to w. If the four alternatives are tones which differ only 
in frequency, then 20210 implies that the difference in pitch between z and 
y is judged by the subject to be greater than the difference in pitch between 
2 and w. The third primitive is a binary relation P on the set of alternatives 
К. P is the relation of strict preference; «Ру holds if and only if the subject 
strictly prefers x to y. 
A numerical interpretation of the relations P and Q is given with the 
following definition of a quaternary utility function. 


DEFINITION 1. Let U be a real-valued function defined over K; then 
U is a quaternary utility function if it satisfies the following conditions for 
every 2, у, 2, and шіп К: 

(а) «Ру if and only if U(x) > Uy); 

(b) zyQzw if and only if U(x) — U) > U(z) — Uw). 


The basic assumption of the model may now be stated. 
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QUATERNARY Hyrornzsis (ОН). There exists a quaternary utility func- 
tion for the relations P and Q and set K, with the following restrictions. 

(a) For every x and y in К, if x ¥ y, then U(x) >“ U(y). 

(b) For every x, y, z, and win K, if U(x) > U(y) and U(z) > U(w) and 
д # 2 огу ғ ш, then U(x) — (у) = U(z) — U(w). 

Restrictions (a) and (b) are necessary if the relations P and Q are to 
have the desired ordering properties, which will be clear after consideration 
of empirieal consequences 1 and 3, presented in the following section. Con- 
dition (b), of course, rules out the possibility that an interval U(x) — U(y) 
equals some other interval U(z) — U(w). However, this exclusion is consis- 
tent with the intended applieation of the model to situations of strict 
preference. 

For every choice that a subject makes, we may write а corresponding 
inequality in utility values (Definition 1). Ап experiment would produce a 
set of inequalities for each subject, and a quaternary utility function exists 
for a particular subject (the model holds exactly) if and only if the resulting 
set of inequalities is consistent, i.e., has a solution. There are well-known 
decision procedures for determining whether or not a set of inequalities has 


a solution [e.g., 6]. 

Satisfaction of the QH is sufficient to guarantee the existence af at OM 
scale, but it is not necessary. This follows from the fact that, as will be shown 
in the next section, it is possible to derive from the QH consequences which, 
if satisfied, guarantee the existence of an OM scale, yet these consequences 
do not exhaust the empirical significance of the QH. This means that the 
QH may be disconfirmed even if an OM scale exists. Hence one important 
problem is to find the weakest conditions imposed by the QH which are 


i roblem i define 
capabl rati тей OM scale. An allied problem is to 
pable of generating the desired terms of the empirical conse- 


progressively stronger alternative models in | › 
quences of ius bane hypothesis, 50 that it will be possible to ed 
how strong а model is sustained by the data. In the oe cabs veo 
of the empirieal consequences which follow from tbe pss МР 
Specified and in the process two alternative models are dehnec. 
Some Empirical Consequences of the Quaternary agen 

By empirical or observable consequence of the an е ч. orem 
quence implied by the QH about the P and Q re ^ " 5 Drops 
fortunately, a result obtained by Scott and Suppes m 1 р к, ше 
that there is по simple set of consequences of the type doe pueden 
completely exhausts the empirical significance of the Q pie 
Stimuli. Derivation of the empirical оойнефпеповв ин А» {зке = 
propositions, such as the independence of these со ДЫҒЫ інің к 
presented, since they are mostly simple, tedious, an 


interesting. 
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The first consequence of the ОН is that the binary relation P is a strict 
simple ordering of K. 'The terminology follows that of Suppes [9]. 


EmprricaL CoNsEQUENCE 1 (Cl). The binary relation P is a strict 
simple ordering of К; P satisfies the following conditions for every x, у, and 
zin K: 

(a) exactly one of the following holds: = = y, «Ру, уРх; 

(b) if zPy and yPz then xPz. 


Thus if Cl is satisfied it becomes possible to derive a ranking without ties 
of the alternatives in K. 

One is perhaps used to thinking of this consequence as the sole axiom 
of an ordinal model, but certain Q relations are predictable from P relations, 
and hence the ordinal model must be defined to include an axiom formulating 
the connection between P and Q where it is possible to predict Q preferences 
from P preferences. Гог example, if three alternatives are ordered «PyP2, 
then this knowledge alone implies that the distance between x and z is greater 
than the distance between 2 and y, and also greater than the distance between 
y and z. This notion is formalized below. 


EMPIRICAL CONSEQUENCE 2 (C2). For every 2, y, 2, and шіп К, 
(a) if zPy and yPz, then zzQxy and xzQyz; 
(b) if zPy, yPz, and zPw, then zwQyz. 


In most preference experiments, one would hardly expect C2 to be dis- 
confirmed if C1 is satisfied, but this confidence might not be at all sustained 
in certain psychophysical experiments. In any event, C1-2 are the axioms 
of the Ordinal Model. 

One further consequence of the basic hypothesis permits the formulation 
of a model which generates an OM scale. Consider a subset P of K X K 
such that for all (x, y) in K X К, (x, y) belongs to P if and only if Py. Р 
then is the subset of K X К which is the set of ordered pairs xPy. Then one 
obvious consequence of the QH is that Q is a strict simple ordering of P. 


EMPIRICAL CONSEQUENCE 3 (C3). The quaternary relation Q is a strict 
simple ordering of P (Q is asymmetric, transitive, and connected in Р); 1.е., 
for every @, y), (2, ш), апа (и, v) in БА 

(a) exactly one of the following holds: (x, у) = (2, ш), xyQzw, or zwQxy; 

(b) if zyQzw and zuQuo, then zyQuv. 


The intuitive statement made in the introduction that a complete 
ordering on utility intervals is required for an OM scale corresponds to the 
formal requirement that Q be a strict simple ordering of P. The restriction to 
P is а restriction to positive differences in utility. 

The fact that C2 and C3 are independent has some interesting impli- 
cations. It will be recalled that the intuitive definition of an OM scale requires 


| 


= 
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only that C1 and C3 be satisfied. However, C2 could be disconfirmed even 
if C1 and C3 were satisfied. Hence C1 and C3 are not sufficient for the deriva- 
tion of an OM scale; C2 is necessary. This is the point of the remark made 
in the introduction that the intuitive definition of an OM scale as requiring 
an ordering of the stimuli and an ordering of utility intervals was inadequate. 

C1-3 do not exhaust the empirical significance of ће ОН. This fact, 
coupled with the fact that these consequences are sufficient to generate an 
OM scale, leads to the important result that an OM scale may exist even when 
the ОН is disconfirmed. Such a model is called а weak ordered metric model 
to distinguish it from the stronger model which assumes the existence of а 
quaternary utility function. Empirical consequences 1, 2, and 3 define the 
Weak Ordered Metric Model (hereafter abbreviated as WOM); i.e., С1-3 are 
the axioms of the WOM. It should be noted that although the term weak is 
used, the model still generates a higher-ordered metric scale. 

Although C1-3 are the set of necessary and sufficient conditions for the 
existence of an OM scale, it is of considerable interest to formulate further 
consequences of the ОН, for а number of reasons. First, completely separate 

t, these consequences could be used for the 


from the problem of measuremen 
predietion of choice behavior. Second, if the ОН is disconfirmed, then it 


becomes possible to state precisely in which way it was disconfirmed by 
examining each of the consequences. Third, the formulation of such conse- 
quences permits the construction of progressively stronger models of choice 
behavior which ean be compared in experimental situations. Finally, the two 
consequences to be presented exhaust the empirical significance of the QH 
under certain special conditions to be discussed after their presentation. The 


first of these consequences is presented below. 

EMPIRICAL CONSEQUENCE 4 (C4). For all 2, у, 2, and шіп К, if zyQzw 
then xzQyw, 100022, and wzQya. 

This consequence is stated in its most general form, but in the usual 
experimental situation the observation «yQzw implies xPy and 2Р0, їп йы 
case the consequence could be simplified to “if ryQzw then azQyw" for e 
predietion purposes. In any event, for any given ordering of stimu ns | 
will have only one conclusion w ith empirical content, not three. Its empirica 


1 1 i lyifa+2> 
ithmetical fact that a > b if and only 
peu 5, timuli are ordered zPyPzPw. Correspond- 


b . For ex: suppose four 8 Weg 
io vhs iced: з. is, writing capital letters до: the d us 
stimuli, “if X — Y»2-W, then X — Z2 E We The om » m 
be written (X — Y) + (Y — 2>4- Zi a ; i pet қ: uality 
be seen that the interval (Y — Z) is common to both ee ud MC 
which can be reduced to X — YZ үш к T dm aa C13 
The following example shows that C4 has implications wht 


do not. Suppose again that four stimuli are ordered aPyPzPw, and further 
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that the six utility intervals are ordered zwQzzQywQzwQzyQyz. P and Q are 
transitive and hence Cl and C3 are satisfied. Inspection of the ordering of 
the intervals indicates that C2 is also satisfied. Note, however, that the 
observation zzQyw implies zyQzw according to C4. But zwQzy was observed; 
therefore C4 is disconfirmed. This means that C4 has implications which 
C1-3 do not. This result confirms the important conclusion already stated, 
that C1-3 do not exhaust the empirical significance of the ОН, in spite of the 
fact that these consequences are sufficient to generate an OM. scale. 
One final consequence of the QH is now presented. 


EMPIRICAL CONSEQUENCE 5 (C5). For all x, у, 2, w, u, and v in K, if 
zyQzw and uxQuv, then uyQzv. 


The empirical content of this consequence is readily seen if the relations are 
written in terms of the corresponding utility values: if X — Y > Z — W and 
U — X > W — V, then U — Y > Z — V. Adding the two inequalities of 
the premise, (U — X) + (X — Y) > (Z — W) + (W — V), which of course 
simplifies to U — Y > Z — V, the conclusion. The arithmetical interpretation 
is (i) ifm > палат > s, then m + т > п + s. In a similar way опе can show 
that C5 also has the arithmetical interpretation (ii) if r > n and m + n > 
т + s, then m > s. 

When an implication of C5 involves four stimuli (e.g., abQcd), then this 
Q relation can also be predicted by C4, although the two predictions may 
differ. Thus one of the subjects in the experiment to be reported made the 
following choices: dfQab, bfQad, cdQbe, and acQcf. The first choice implies 
the second by C4, but the third and fourth imply adQbf by C5, which con- 
tradicts the second. This shows that C5 has implications which C1-4 do not, 
since the total set of observations was consistent with СІ-4, but C5 was 
disconfirmed. 

If a O5 implication involves only three stimuli (e.g., vyQyz), then it 
cannot be predicted by C4. Tt also follows that in such a case the premises 
of C5 must involve only five stimuli, not six. Therefore the number of pre- 


dictions which ean be made by C5 but not by C1—4 cannot exceed al where 
т is the number of stimuli in К. However, even if a C5-implication is not 


predictable by C4, it may be predictable by C3. Since there are () ordered 


utility intervals, there will be exactly () — 1 Q relations which are not 
predictable by C3. Therefore the maximum number of predietions which can 


be made by C5 but not by С1-4 is м ог () — 1, whichever is smaller. This 


upper bound reflects the maximum additional predictive power of a model 
which contains C5 as an axiom as well as C14. 


Although C1-5 do not exhaust the empirical significance of the QH for 


{Р 
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all finite sets of stimuli, a weaker statement on sufficiency conditions can 
be made: if restriction is made to predictions based on not more than iwo 
choices (i.e., of the form “if А then В” or “if A and B then С”), then C1-5 
do in fact exhaust the empirical significance of the QH. This means that 
under such a restriction, any prediction which сап be made with the ОН 
can also be made with С1-5. The increase in predictive power resulting from 
the utilization of more than two choices is slight for small sets of stimuli 
(less than ten), and hence this is an important result. 

In formulating the WOM, the set of necessary and sufficient conditions 
for the existence of an OM scale have been specified. Such a model may be 
compared in experimental situations with the Strong Ordered Metric Model 
(hereafter abbreviated SOM), which assumes the existence of a quaternary 
utility function. Such a strong model is perfectly satisfied if and only if the 
set of inequalities in utility values corresponding to the set of observed P and Q 
relations has a solution. However, аз pointed out, even if the solution does not 
exist, the WOM may be satisfied, and hence an OM scale may exist. 


Relation to Two Other Theories 

The present model w relation to two other theories. 
The first is that used by Siegel [8] to obtain an OM scale, and the second is 
a model of riskless choice developed by Adams and Fagot [1]. 

The essential device underlying Siegel’s approach is а one-person game 
jn which the subject chooses between two options, each of which is a prob- 
ability combination of two outcomes. A chance event Ё, which has an ex- 
perimentally determined subjective probability of one-half, is defined. 
The subject is required to choose between two options (х, y) and (г, w). 
If he chooses the first and Æ occurs he gets т, while if Æ does not occur he 


gets y. The second option is determined in a similar manner. 
If one now introduces a quaternary relation № such that ayRzw holds if 


and only if the subject chooses the option (т, y) when presented with a choice 
between (x, У) and (2, w), then condition (b), Definition 1, of а quaternary 
utility function may be modified as follows: 
(b/) тугш if and only if U(x) + U'Y) > сқа) + U'(w). 

n individual makes choices 
ying to maximize expected 


rill be discussed in 


Condition (b’) is based on the assumption that & 
among alternatives involving risk 28 if he were tr 

ility 2-213). 4 
js сат thods for deriving ОМ scales, and it would be of 


'Thus there are two me › a 
considerable interest to determine if the same scales would be ы E nr 
both methods. The following hy ostulates à relationship betwee 


R and Q. 


ВО HyporuEsis. F 
vwQzy. 


pothesis p 


or every 2, V» 2 and w in K, xyRzw if and only if 
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'This hypothesis is equivalent to the assumption that the utility func- 
tions U and U’ are the same. Given a set of axioms on Q (such as С1-5), 
these axioms can be immediately transformed to a set of axioms on R, by 
means of the RQ hypothesis. It then becomes possible to determine if indi- 
vidual axioms hold for both Q and R, rather than simply testing the stronger 
RQ hypothesis. It should be noted that the existence of U’ is not necessary 

_ for the derivation of an OM scale; satisfaction of axioms equivalent to C1-3 
is sufficient. 

There are several interesting possibilities which could arise from ex- 
perimental comparisons of the two methods. For example, both U and U’ 
may exist, yet not be the same. Or, neither U or U' may exist, yet an OM 
scale may exist for both (i.e., the WOM may be satisfied for both Q and R 
observations), and this scale may not be the same. Finally, one method may 
produce an OM scale, but not the other. 

Another theory, formally quite similar to the present model, is a theory 
of riskless choice developed by Adams and Fagot [1]. This theory is concerned 
with subjects’ choices between alternatives which are multidimensional, 
although the theory has been worked out only for the two-dimensional (or 
two-component) case. Such alternatives might be political candidates who 
are described as varying in two characteristics, liberality and foreign policy, 
for example. In this case the subject would be required to state a preference 
for one of every pair of such candidates. Or, in an extension, the alternatives 
may be pairs of objects of any sort. For example, one two-component alterna- 
tive could be book z and book y, and the subject might have to choose between 
this alternative and a second which consisted of two books z and w. It is clear 
then that the components of the two-component alternatives can be thought 
of as belonging to a set K of alternatives, such as has been considered previ- 
ously in this paper. The model assumes that the individual behaves as if he 
assigns subjective values (utilities) to each of the components independently, 
and then adds the values together to get the value of the composite alternative. 
The fundamental assumption of the model is the hypothesis of the existence 
of this additive utility function. Comparable to condition (b), Definition 1, 


(b^) @,#Р@, w) if and only if U"() + UY) > U"() + 070, 
where P is the relation of strict preference, 


The model generates an OM scale, and a simple transformation relates 
this model to the present model. This relationship is indicated by the follow- 
ing hypothesis. 


PQ Hyroruesis. For every т, y, z, and w in k, (x, y)P(z, w) if and only 
if zzQuy. 


This hypothesis is equivalent to the assumption that the utility functions 
U and U" are the same. All three models are related by the following corollary. 
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Совотллну or RQ лхо PQ HYPOTHESES. For every т, y, z, and w in К, 
(е, y) P (z, w) if and only if тугш. 


It should be realized that E holds be 
alternatives. Thus in the case of P the outcome 
of В the outcome is 2 or y, depending upon the outcome of a chance event E. 

In this additive model of riskless choice, the basic assumption of the 
additivity of the components is usually used simultaneously to determine the 
utility values and to make predictions. Under these conditions, the predictive 
consequences of the model are much weaker than those using utility values 
obtained outside the model by independent methods of measurement. 
Observations on Q or Ё relations could provide this independent method of 
measurement. In other words, U or U' could be used to assign utility values to 
the alternatives in a basic set К, and then the additive model could be used 
to make predictions between pairs of these alternatives, under the assumption 


that the utility values are additive. 


tween options and P holds between 
is т and y, whereas іп the case 


Some Experimental Results 


Some results of an experiment on the utility of course grades afford a 
test of the model and illustrate the derivation of an OM scale. Ten students 
їп introductory psychology served as subjects. The alternatives in the set K 
are the course grades A, B, C, D, and Ё. Appropriate operational definitions 
were given to the relations P and Q such that aPb means that the subject 
prefers the grade А to the grade B, and dfQab means that the difference, in 
value (utility), between the grades D and Г is greater than the difference 
between the grades A and B. Care was taken to insure that each subject 


thoroughly understood the operation of comparing two utility intervals and 
reporting the inter 


val which was subjectively greater. 
Let n be the number of stimuli. 


Then № , the number of Q relations 
predictable from à knowledge of an ordering of the stimuli, is 
n т 
(1) No zz xt) F ("). 
{ all comparisons are made, 


in K; i 
14 consist of 10 P relations and 45 


the complete data for ; 

О relations. Of these 45 observations, 25 would be predictable from C2 

(№ = 25). However, in à preference experiment of this к MEN a. 

little necessity for making 8 complete test с rg aus ты % Бе) Е 
" to each subject. The resp 

such comparisons were presented ested; it was assumed 


j Y istent with C2. 
the ten subjects were consiste wwe umo PePdPÍ. 


that all subjects preferred the grades i in pete 
biects v es 
f the subjec + be predicted from а knowledge 


there are five stimuli 


In this experiment 
each subject wou 
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of an ordering of the stimuli, and hence these observations contain the 
ordered metric information. 

"These choices are divided into subsets by means of C4; subset В is pre- 
dictable from subset А by means of C4. Utilizing this consequence one сап 
predict (16) from (2), (17) from (5), (18) from (3), (19) from (12) and (20) 
from (8); thus C4 is perfectly satisfied. 


TABLE 1 


Observations of One Subject in Experiment on Utility of Grades 


Subset A Subset B 
l. abQbc 6. bfQab ll. acQcd 16. acQbd 
2. abQcd 7. bcQcd 12. dfQac 17. bfQac 
3. dfQab 8. dfObc 13. cfQac 18. bfQad 
4. bdQab 9. cfQbc 14. adQdf 19. cfQad 
5. cíQab 10. dfOcd 15. dfObd 20. cfQbd 


e MM M——ÀM— —À --. 


А necessary and sufficient condition for admission of an observation to 
subset A is that the two intervals corresponding to the two ordered pairs of 
а Q relation do not overlap. For example, if vPyPsz, then the only such relation 
is vyQyz (ог 12020, whichever is observed). The number of Q relations in 
subset A is 


(9) а P i » 


The two intervals for а О relation in subset B always overlap. For 
example, observation (16) implies A — Ç > B — D, and it is clear that these 
two intervals have the interval B — (in common. The number of observations 
in subset B, and hence the number of predictions that can be made in any 
experiment by means of C4, provided that certain observations are made in 
subset A, is 


@) к» = (3) 


It is important to note that although N, choices can be predicted from certain 
choices in subset A, these predictions cannot be made by the WOM, since 
C4 is not an axiom in this model. 

Inspection of Table 1 shows that the choices of this subject were con- 
sistent with C3 (Q is transitive), and with C4. He satisfies not only the WOM. 


аа енеке 
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but also the SOM (the set of inequalities corresponding to the observations 
have а solution). The OM seale which is derived from the observations in 
Table 1 is defined by the following ordering of utility intervals 


(4) A—EN6izEBePBQ—FSA—DID-—ESAcO 
B-D>A-B>B-C>C-D. 


In general, if there are 2 intervals, then (¢ — 1) Q relations are sufficient 
to define the OM scale—but not always necessary. For example, of the nine 
Q relations corresponding to (4), the following seven are sufficient to generate 
the OM scale by the WOM: cfQad, adQdf, dfQac, acQbd, bdQab, abQhe, bcQcd. 
The relations afQbf and bjQcf are not necessary, since they are implied by 
the P relations by means of C2. Thus the remaining 38 @ relations contain 
superfluous information since they are predictable from these seven by means 
of the axioms of the WOM, thus providing a test of the model. Depending 
on the structure of the scale, as few as three observations may in some cases 
provide sufficient information for the construction of the OM scale. Un- 
fortunately, precisely which observations provide this information cannot be 
specified in advance. However, Siegel ([4], р. 141) has developed a “maximin” 
method which maximizes the number of predictions which may be made from 
observing a minimum number of choices, and specifies the order in which 
choices should be examined in constructing the scale. His method may be 


adapted for use with the present model. 
The number of observations necessary 
entire set of observations can be taken as a measure of the predictive power 
The smaller the number of such observations, the greater is the 
e model. In the case above, two of the observations, 
in the SOM since they are implied by 
Thus in the SOM only five observations 
I seven are necessary, indicating the 
s experiment 


and sufficient to represent the 


of the model. 
predictive power of th 
cfQad and acQbd are not necessary 
d[Qac and abQed, respectively, by C4. 
are necessary, whereas in the Wo) 
correspondingly greater predictive power of the former. In thi 
the number of choices necessary and sufficient to represent the total set of 
45 observations ranged, in the WOM, from a low of six for one subject to a 
high of eight for another subject, and in the SOM, from a low of three to à high 
of five, the lower values of the SOM being indicative of its greater predictive 
power. This concept of predictive power is treated more extensively in other 


sources ([5]; [3], рр. 28-86). 


The general results of the 
1. Seven of the ten subjects perfectly satisfied the WOM; hence for these 


j i i { f I scale. 
seven subjects it was possible to construct an OM | | я 
2. Two of these seven failed to satisfy the SOM. These tw жесек 
are of special interest, since for them it was possible to epi Е = 4 
in spite of the fact that they did not satisfy the ОН. One of these t 


experiment are as follows. 
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single disconfirmation of C4, and the second had a single disconfirmation 
of C5. 

3. Seven of the ten subjects perfectly satisfied C4 (five times each); 
the remaining three each had a single disconfirmation. 

4. At most one prediction for each subject can be made by C5 which 
cannot be made by means of С1-4, since there are only five stimuli in this 
experiment. The results were that one such prediction was made for each of 
three subjects, and one of these predictions was disconfirmed. (If the number 
of stimuli were increased by one to six, then the maximum number of impli- 
cations of C5 for each subject would increase from one to six.) 

5. For each of the three subjects who did not perfectly satisfy the WOM, 
the reversal of a single choice would have resulted in satisfaction of the model; 
however, one of these three still would not have satisfied the SOM. Methods 
of analyzing the fit of the model have been devised, but discussion will be 
deferred to a later paper on the analysis of a series of such experiments. 
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A NOTE ON FACTOR ANALYSIS: ARBITRARY 
ORTHOGONAL TRANSFORMATIONS 
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A modification of the Gram-Schmidt process yields an easily constructed 
orthogonal transformation matrix which may i 


by rotating a centroid factor matrix into an abbreviated bi 
general factor being defined as the centroid of a specified subgroup of reason- 


ing tests. 


Starting with an n by m orthogonal factor matrix A, such as a centroid, 
principal axis, or maximum likelihood matrix, with entries а representing 
the loadings of tests or other initial variables on m factors, it is decided 
on substantive (rather than mathematical) grounds that one rotated axis 
should pass through а particular point. The other axes may be anywhere, 
so long as they remain orthogonal to the one chosen. The problem is to 
construct an orthogonal transformation matrix to accomplish this purpose. 

Reyburn and Taylor have discussed this problem [2], and have presented 
a method for constructing the desired transformation matrix. Their method, 
however, is fairly complicated, though less so, as they point out, than the 
direct application of the Gram-Schmidt process. By taking advantage of 
the fact that all of the axes but one may be defined arbitrarily, the Gram- 
Schmidt process may be simplified to the point where the desired trans- 
formation matrix can be computed with much less effort than is required 


by the Reyburn-Taylor procedure. қ х 
The general Gram-Schmidt process converts any matrix X into an 
r collinear with the first 


orthogonal matrix Z whose first row is а unit vecto 
row vector of X. If the first row vector of X is a unit vector, the first row 
vector of Z is identical to it. In this case the general Gram-Schmidt equa- 


tions are 
Z= Xi; 
2% [Xs 3 (Z,X9Z/Ks D 


Z, = [Xs — X9. — (2 XXE » 
— (2% -Х9274/Ж. 2 


= (Z3:Xs)Zs = (2.-Х)2./Кь Й 


Zr = (2,-Х4) 7, n (Za: X925 


[X; — (£i X34. — (2, Хә)» 


Zs 
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etc. The 2; and X; are the row vectors of Z and X, the Z;- X; in parentheses 
are scalar products, and the K; are normalizing factors. 
Let the selected unit vector be a, Б, c, а, e, --- , and let X be of the form, 


b cde 
0000 
1000. 
0100 
00010 


The scalar products of the Gram-Schmidt equations then become single 
elements of Z, and 


Z 2X, 

Z: = ІХ. — а2,/К,, 

Zs = [Xs — bZ, — 2,74/К,, 

Z, = [X, — cZ, — 237, — 742-/К., 

Zs = [Xs — 427, — 2..0, — 242, — 742./К,, 


etc. These equations сап be solved literally for the two-variable, three- 
variable, four-variable, and five-variable eases. First define 


а 
1 
Х-|0 
0 


k-1-—s, 
k-1—-a-—VUW, 
k=l- čte. 


These k’s are related but not identical to the K’s of the previous equations. 
The desired transformation matrices are then 


Z= Га i , 
Lb —a 
[a b с 
Zu М —ab/k, —ae/k,|, 
0 с -Ьһ, 
Га b c d 
es ka —ab/k, | —ac/k, —a d/k, 
0 k./k, ^ —be/k,k, —b а/к, |? 


Lo 0 а/Т —c/k; 
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a b с а е 

ka —а/һь —ac/kz —a d/k; — ae/ ks 
2-|0  k/k  —b/kk, —bd/kk, —be/Koks | 

0 0 kk — —cd/khk, се, 

0 0 0 e/k, —d/k, 


The system by which larger matrices may be constructed will be clearer if 
it is noted that kı = 1 and that the final k of each matrix (not shown as such 
in the last row of each of those above) is equal to the.last element of the 


first row. Thus for the 5 by 5 matrix, kè = 1—a — 0 —c — d = è, and 
ks = e, since Z, is a unit vector. With this notation, 
a b 6 а е 
kk, —ab/kıkz —ac/k,k, —a d/kıkz —ae/kkz 
Z= 0 ks/ ks —bc/k,k, —b d/kzką - be/ksks 
0 0 kk, фе аа —ce/kska |” 
0 0 0 ks/kı  —d e/kıks 


and for the two-, three-, or four-variable case only the first two, three, or four 
rows and columns are used.* The required transformation is then 


AZ! =G, 


G being the rotated factor matrix. 
Example 

The A matrix of Table 1 is a centroid matrix computed by the writer 
from a table of intercorrelations reported by Swineford ([3], p. 14, Table 4). 
Her subjects were 504 school children of both sexes in grades 5 through 12, 
all between the ages of 12-0 and 14-5, and all from the Chicago metropolitan 
area. The centroid solution was iterated ten times to obtain communalities 
all stable within = .001. 

Swineford argues on substantive (psychological 
factor can be defined by the first three tests, whic! 
writer to be reasoning tests. Ac 
will be rotated orthogonally so th 
tests 1, 2, and 3. The first row of 
the normalized sums of the first thr 


) grounds that a general 
h appear to the present 
t, the centroid matrix 
at one axis passes through the centroid of 
the transformation matrix Z consists of 
ee entries in each column of A. The rest 


of the transformation matrix is constructed by the Gram-Schmidt formulas 
i i ick t of Mathe- 
*I debted to my colleague Dr. Frederick A. Ficken, Department ог. 
matics, Аю TIRE Tonn sei, for a proof that Z, as defined by this last matrix, is actu- 
ally orthogonal. 


cepting her argumen 
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for the three-variable case. Thus 
-9059 2405 —.3485 


2- jus — 5 7455 


£ —.8229 —.5679 


TABLE 1 


Factor Matrices (Swineford Data; N= 504) 


Name of test A —z c ғ 5 ? 
an а: “g h Бі 8:2 біз fi Фа fa fo uo з № 
l.Arith.Cosput. 659 285 -308 610 773 -097 -060 773 -111 -026 677 000 000 458 
2.Series Complet. 643 065 -243 477 683 058 085 683 101 -019 688 ооо 000 473 
3.Deduction 619 160 -188 444 665 040 -025 665 Oll 046 661 000 000 437 
&.бепега1 Inf. 802 228 235 750 699 398 -321 699 054 508 757 000 415 745 
S.Reading Comp. 730 202 106 585 673 284 -226 673 041 361 718 000 265 586 
6.Word Meaning 756 261 397 797 609 482 -k40 609 озо 652 668 000 622 833 
7.Punched Holes 716 -512 -085 782 555 503 470 555 688 023 578 652 000 759 
8.Dravings 660 -420 029 613 487 57 329 487 598 133 537 577 000 621 
9. Vis. Imagery 638 -266 057 481 494 450 187 494 050 .186 554 413 000 47 
Su» 6223 003 000 5539 5638 2635 -001 5638 1862 1864 5389 

Check 5638 2634 -002 5638 1863 1864 | 

| 

Note: All entries have three decimal places. a | 

Postmultiplying А by Z' is equivalent to multiplying А by Z, row by | 


row. Doing this, the entries in matrix G of Table 1 are obtained. Those 10 
the а column are the loadings on the general factor defined by the centroid 
of tests 1, 2, and 3. These tests have small loadings summing to zero within 
rounding error in columns g;; and 9:3 , as they should. For the rest, columns 
gi, and gis represent a residual centroid matrix; all entries in column 42 7 
except those of the first three tests are positive and substantial, and those in 
column gia sum to zero within rounding error. If this were not already ® 
residual centroid matrix, it could be made into one by normalizing the sums 
of the last six entries in columns 0: and g;; , using these values as the first 
row of а new two-variable Z, completing this Z by the Gram-Schmidt formulas, - 
and postmultiplying columns ф and gis by 27. С 

То complete the derived bifactor solution, the residual centroid should be 
located at the center of the positive quadrant. Hence the axes of 0:2 21 
gia should be rotated so that the new axes will be at 45 degree angles to e 
first residual centroid axis. To do this one employs the two-variable Landah 
transformation [1], 
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Postmultiplying columns g;, and g;, of GI by L’, yields the columns f; and 
Та of matrix Р, Table 1. Column fa is identical to column 0:4 of matrix G, 
and matrix P is the derived bifactor solution. Matrix 5 is Swineford's original 
bifactor solution, computed by Holzinger’s methods. 

Considering first the similarities between the two solutions, it is clear 
that the first factor is a general factor, the second factor a space factor, and 
the third factor а verbal factor. So far as gross factorial interpretation is 
concerned, the two solutions are essentially equivalent. 

А Considering the differences between the two solutions, it is the writer's 
opinion that the derived solution is superior on three counts. First, it uses 
all of the common variance. This property, admittedly, depends upon the 
fact that the centroid solution was iterated, a tedious procedure at best. 
But the centroid communalities sum to 5.539, while the direct bifactor 
communalities sum to only 5.389. The distributions of the third-factor 
centroid residuals (at the tenth iteration) and the direct bifactor residuals 


appear below. 


070 050 030 010 -00  —030 —050 070 —090 —110 


Residual: 089 069 049 029 009 —01 —031 —051 -071 —091 
Centroid: 1 9 18 6 2 
ré 3 14 5 3 1 1 


Bifactor: 1 1 7 


The centroid method clearly gives a better factorial fit. With larger matrices, 
moreover, the number of iterations required to stabilize the communalities 
is much reduced. 

The second reason for preferring a derived solution is that in it errors 
are averaged over whole columns of the correlation matrix, and distributed 
over all factor loadings, both before and after rotation. In the direct bifactor 
method, near-zeros become exact zeros, and all error variance is forced into 
the nonzero loadings. 

The third reason for preferring a derived solution is that it shows the 
secondary relationships more clearly. In columns f;; and f;; of F, all entries 
other than the first three are positive. Hence it is clear at once that there 
is some variance common to the verbal and space factors over and above 
What they have in common with the reasoning-general factor. In the direct 
bifactor solution, the general factor is not wholly defined by the three reason- 
ing tests; it includes also most of the variance common to the space tests 
ani the verbal tests. A certain amount of additional “forcing” is necessary 
to achieve this result; it cannot be produced by rotations in 3-space without 
Moving the general-factor axis away from the centroid of the three reasoning 
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tests, in which case these tests acquire larger loadings on the other two factors. 

This is, in fact, an abbreviated bifactor solution. A complete bifactor 
solution ([4], ch. XIX) requires a general factor in addition to as many group 
factors as there are simple-structure factors. The general factor is then 
defined by the correlations between the primary factors of the oblique simple 
structure rather than by any one subgroup of the tests, and the general-factor 


loadings are obtained by projecting the second-order factor loadings back 
into the first-order domain. 


REFERENCES 


[1] Landahl, H. D. Centroid orthogonal transformations. Psychometrika, 1938, 3, 219-223. 


[2] Reyburn, H. A. and Taylor, J. G. Some factors of personality. Brit. J. Psychol., Gen. 
Sec., 1939, 30, 151-165. 


[3] Swineford, F. A study in factor analysis: the nature of the general, verbal, and spatial 
bi-factors. Supplementary educ. Monogr., 1948, No. 67. 


[4] Thomson, С. Н. The factorial analysis of human ability. (3rd ed.) New York: Houghton 
Mifflin, 1948. 


Manuscript received 6/5/58 


1, 


PSYCHOMETRIKA—YVOL, 24, No. 2 
JUNE, 1959 


RANDOMLY PARALLEL TESTS AND LYERLY'S BASIC 
ASSUMPTION FOR THE KUDER-RICHARDSON FORMULA (21) 


Тверевгс M. Lorp 
EDUCATIONAL TESTING SERVICE 


А Тһе К-В (21) formula can be derived without recourse to the unde- 
sirable assumption that the test items are all indistinguishable. 


Certain of the statements made by Lyerly [4] about the assumptions 
underlying the K-R (21) formula will mislead readers to reject it on the 
grounds that the assumptions are rather implausible.* The present note is 
written in the hope of preventing such a conclusion. 

Lyerly ([4], р. 269) states "The basic assumption for K-R (21)... 
implies that individual items have no separate identities, even though each 
may be worded differently or may ask a question different from that asked 
by any other item. Г urthermore, item equivalence from person to person 
does not exist. The same item, worded identically, may appear on everyone’s 
test booklet, but item 1 for Subject A is regarded as being no more like item 
1 for Subject В than it is like item 2 for Subject B.” A few sentences later, he 
asserts that Lord's derivation [1] “implicitly uses this assumption." 

It isa fact that IK-R (21) can be derived from the fundamental assumption 
just quoted. It does not follow that the derivation of K-R (21) in [1], or other 
derivations, "implicitly use" this assumption, or imply this implication. 
K-R (21) was originally derived on the assumption that all the test items 
were equivalent (of equal difficulty and discriminating power); it does not 
follow that the derivation in [1] is based on the same assumption, although 
this too has been asserted. If two sets of assumptions lead to the same con- 
clusions in all possible applications, then they may be considered, in a sense, 
identical assumptions; but the fact that they lead to the same conclusion 
in some particular case does not prove that the assumptions are identical, 

" the ir implications are the same. ) 
“| Тек ке of view, the derivation of K-R (21) in [1] does not rest 
on "assumptions" at all. The derivation simply asserts me с | 

Prepare а very large pool of dichotomous achievement ог 2. items 
of any kind or kinds whatsoever. Draw randomly and independently from 


i i inary mental 
Jhi erly’ i ау seem rather implausible for ordinary men 
ee es корта DUNS for captain Duy үң А peti a Pune 
$ Б) Е y j d by the same judges. Lyerly в re: 
A bees БОШ зора ү ние атр note deals only with the usual type of 
objective test, as do the references cited. 
175 
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this pool à number of samples of n > 2 items each. Each sample will be 
treated as an n-item test, administered (with ample time limit) to а group of 
examinees, and scored by counting the number of right answers for each 
examinee. (In actual practice special measures will be needed to minimize 
fatigue and practice effects, and also other types of variability within the 
individual examinee; these admittedly are not taken into account by the 
derivation, but they are irrelevant to the point at issue here.) 

Now, take the scores of a particular examinee on the various n-item 
tests and compute an unbiased estimate of their standard deviation. This 
is his estimated standard error of measurement. Obtain 
precisely, the quadratic mean) of the estimated individu 
of measurement. What the derivation in [1] asserts is that the average standard 
error of measurement so obtained is the same (except for sampling fluctua- 
tions) as that obtained from K-R (21) for any of the n-item tests by the 
usual formula 


8. E. Meas. = s, VI — "EN 

where r.. is K-R (21). 
The foregoing assertion, 
viding fatigue, practice effect, 


ап average (more 
al standard. errors 


made in [1], is experimentally verifiable, pro- 
and other temporal variability in the examinee 
can be minimized. If temporal effects are eliminated from consideration, so 


that the order in which the items are administered is immaterial, then it will 
make no difference whether the various n-item tests are drawn by random 
sampling in advance of the testing, or whether the pool of items is administered 
first and the random samples of n items drawn afterwards. In the latter case, 
the experiment is identical with that performed in an elementary Statistics 

hite balls from an urn. The results that will 
уп since the time of Jacob Bernoulli. 


test may be appreciably easier than 
contain several items that are unusu: 


"randomly parallel? tests. In 
ted by random sampli 
ably be better than reliance on a 
nd, neither are tests constructed so 
ding to the assumptions underlying 
‘parallel” forms usually have slightly 
orth. Although neither mathematical 


ы -" - CEG 
- uu — c vw O "Qs 
—P ЖЕР». 7-2. А. у 
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model describes exactly any actual pair of so-called "parallel" tests, both 
models are very useful in mental test theory. The utility of the randomly 
parallel test model is evidenced by some of the practical conclusions to which 
it has led, as shown in [2] and [3]. Tt is desirable, therefore, that this model 
should not be identified with the assumption given in quotes at the beginning 
of this note. 
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ESTIMATING ITEM INDICES ВУ NOMOGRAPHS* 


Вовент M. COLVER 
DUKE UNIVERSITY 


., ,Two nomographs are presented for estimating item validity indices 
identical in value to those obtained from Flanagan's table and to those 
obtained from Davis' chart. Experience has shown that the use of the nom- 
ographs results in the saving of a significant amount of time with no loss in 
accuracy. The nomographs also provide а method of quick conversion between 
the familiar coefficients and the Davis indices, which are less familiar but 


which offer greater flexibility. 


s of a test, the difficulty and the dis- 
have significant bearing. The 
t determined by these 


Tn evaluating the effectivenes 
criminatory power of the component items 
usability, reliability, and validity of the test are in par 
characteristics. 

Some of the more common statistical indices currently in use to evaluate 
test items are: critical ratio, phi coefficient, biserial correlation, point biserial 
correlation, tetrachorie correlation, and product moment correlation. 
Although these indices are based on different underlying assumptions, all 
involve computational proccdures that become laborious. The caleulation 
needed for the necessarily substantial number of items in a test often repre- 
sents more time and effort than are available to many test makers. 

Two general methods have been developed to reduce this computational 
labor: graphic methods (nomographs and Abacs) for estimating the various 
coefficients, and tabular presentations of coefficients and indices. These tables 
are based primarily on procedures outlined by Kelley [9]. 

Nomographs for estimating critical ratios or /s were developed by 
Appel [1] and Arnold [2]. Dunlap [4] modified the standard Dunlap and 
Kurtz Nomograph [5] to estimate biserial correlations for test-item analysis 
purposes. Kuder [10] devised a set of nomographs to estimate point biserial r, 
biserial r, and four-fold correlations. Lawshe [11] presented а nomograph, 
based on the Kelley procedures, for estimating the discriminatory values of 


test responses using & scale of *D-values" of T 5 to - 5. А 
Guilford [8] constructed an Abac for estimating the significant (and very 
Fan [6], in developing an item analysis 


significant) levels of phi coefficients. 


ж) Robert M. Colver, 1959. 
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i f either of these Abacs 

d an Abac, as did Flanagan [7]. The use o 1 " 

A diee would be diffieult because of the complexity of their 
diagrams. ! ) 
са the most widely used techni 
reading of tables based primarily on Kelley 
most satisfactory item validity index is th 
table ([13], 


ques are those involving the 
's procedure. Probably the one 


rity of his 
* the numerical values of 
entirely unfamiliar to the user and will 
tation on that account,” 


: they also provide a meth 
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familiar but which 
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an upper group and 


ntage of success on a test item of 
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€cess on a lower group are needed, 
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As Kelley [9] has shown, these groups should be based on the higher 27% of 
the total sample and the lower 27 o of the total sample. (In entering the 
difficulty nomograph these particular percentages are recommended simply 
because they are needed to enter the discrimination nomograph.) 

The percentage of success of each of the two groups can be determined 
by calculating the per cent of each group that marks the correct answer, but 
Davis ([12], р. 277) has shown that the number of examinees that actually 
knows the answer is a more fundamental estimation; therefore, it is recom- 
mended that the percentages of success of each group be determined by the 
formula suggested by Davis ([3], p. 6): 


Е — ДК — 1) 


ud N-NR ^" 
where 
P = the proportion of the sample that knows the answer to an item, 
R = the number of testees in а sample that answer the item correctly, 
W = the number of testees in the sample that answer the item incorrectly, 
N = the number of testees in the sample, 
NR = the number of testees in the sample who do not reach the item in the 
in the time limit, 
K = the number of choices in the item. 


The Discrimination Nomograph (Figure 1) 


To obtain the product moment correlation or the item discrimination 
index locate the determined percentage of success of the higher group on 
scale Ру and the determined percentage of success of the lower group on 
scale P; . Lay a straight edge (a transparent one is preferable) through these 
points. The discrimination indices are then read at the point where the 
straight edge crosses the center scale. The right side of the scale, r, gives 
product moment correlations and the left side of the scale, 0,1, gives the 
discrimination index. 

A product moment correlation, determined by any method, can be 
directly converted to a discrimination index by locating the г value on the 
r side of the center scale and reading its corresponding value on the DI side 
of the center scale. Similarly the 257 values can be converted directly to r 


values. 
The Difficulty Nomograph (Figure 2) 

To obtain an estimate of the percentage of correct responses in the 
entire sample and the difficulty index locate the determined percentage of 
success of the higher group on scale Ру and the determined percentage of 
Success of the lower group on scale P; . Lay a straight edge through these 


Pu 
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points. The indices are then read at the point where the straight edge crosses 
the center scale. The right side of the scale, p, gives an estimate of the per- 
centage of correct responses in the entire sample and the left side of the 
scale, D I, gives the difficulty index. 

` Direct conversion of percentage of success, determined by any method, 
to the difficulty index can be made by locating the value of р on the р side of 
the center scale and reading its corresponding difficulty index on the DI side 
of the center scale. Similarly a difficulty index сам be converted directly to 
an estimated percentage of success of an entire sample. 


Other Uses and. Limitations 


It has been suggested ([12], p. 298) that “Tlanagan’s table has general 
utility and may be found exceptionally useful in many instances outside the 
field of test construetion where economical approximations of biserial cor- 
relations are required.” Similar use could be made of the r’s obtained from 


this discrimination nomograph, as the values are identical to those obtained 
from Flanagan's table. 


The literature on the use of Flanagan’s and Davis’ tables points out some 


limitations for each table. Similar limitations would apply to the use of these 
nomographs. 
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BOOK REVIEW 


Oléron, Pierre. Les Composantes de l'Intelligence d’aprés les Recherches Factorielles. Paris: 
Presse Universitaires de France, 1957. Pp. viii + 517. 


One of the important consequences for psychology of the founding of a quantitative 
Science of biological variation by Francis Galton and Karl Pearson was that Charles 
Spearman, in 1904, could present to the field a paper entitled "General intelligence ob- 
jectively determined and measured." This paper not only resulted directly in the establish- 
ment of factor analysis as a new discipline in psychology, but it is indirectly responsible 
for Psychometrika and, we must acknowledge, for the livelihood of quite a number of 
people who read this journal. In the more than fifty years of research and controversy 
which the paper initiated, factor analysis has become widely accepted as a general method 
for the study of multivariate relationships and has contributed to the theoretical develop- 
ment of modern statistics. At present the original issues in factor analysis are largely 
resolved and the field now seems to be in a period of consolidation before entering a second 
phase. M. Oléron has chosen this very opportune time to present a complete review of 
contributions in the first phase and to suggest possible directions for research in the second. 
Some indication of the thoroughness of the review is given by the length as well as by 
the 471 item bibliography and useful descriptive list of 157 psychological tests. The ap- 
proach is largely historical, somewhat like Dael Wolfle’s Factor Analysis to 1940 (Psy- 
chometric Monograph No. 3, 1940), and deals with issues and results without going deeply 
into methodology. The only recent comparable work in English, Sten Henrysson’s Ap- 
plicability of factor analysis in the behavioral sciences, (Stockholm, 1957), is less detailed and 
more technical. A similar but less current work is P. Vernon’s, Structure of Human Abilities. 
(New York and London, 1950). 

The review is organized under three main headings: the two-factor theory, the 
multiple factor approach, and results. The first two sections are developed largely in 
terms of the running controversy between the advocates of the two-factor theory of human 
ability, with Spearman at the forefront, and the opposition headed by Godfrey Thomson 
and L. L. Thurstone. The author follows the published record of the dispute closely, re- 
producing the arguments and counter arguments of the period, but not adding much in 
the way of original criticism. Especially in certain technical matters, such as the assump- 
tions about the null distribution of tetrad differences and residual correlations used in 
the tests of the number of significant factors, some definitive criticism based on modern 


statistical theory would have been valuable. The treatment of the two points of view in 
the controversy is eminently fair, but it becomes apparent that the author favors the 


Thurstonian methods because of their greater generality. : 4 қ 
The section on results collects the widely scattered literature of factorial studies 


nized summary of major findings pene E даана 

i i oted in some cases, but extensive numerica ulation 
е; з прое К comments in this section are more extensive and 
help the reader considerably in judging to what extent these findings contribute to a co- 
muri o Mere po is the question of what the factorial studies 
contribute to the concept of intelligence. Factorists кашу a aa d et Ta 
cussing intelligence, but the wide influence of the Binet scale and the "in АП pede о 
tient" in mental testing practice makes this difficult. Both o ous Ame СЫ 
eventually took positions on the question, and they are € Bees T De 
ned points a a ыза ш е Е Rin a ho stri етің factor approach 
though only as minor sources of test variability. Similarly, the stric : 


into а well-orga 


187 


188 PSYCHOMETRIKA 


^ 


of Thurstone was compromised when the 
"Thurstone interpreted the second- 
ability acting in varying degrees 
emphasis on the group factors, 
Stood in a useful way when the 


appearance of group f :огз, and 
depends upon the arbitrary choice 
niquely determined. As a consequence, factors 
ез are more minutely "pecialized, and there is 
mon perspective. Various attempts to organize 
rtance in practical terms, such as academic suc- 
ected as inconclusive and 
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ITEM SELECTION PROCEDURES FOR ITEM VARIABLES 
WITH A KNOWN FACTOR STRUCTURE* 


С. ELFVING 
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В. SITGREAVES AND H. SOLOMON 
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This paper discusses the item selection problem when the item re- 
sponses follow a linear multiple factor model. Because of this restrictive 
assumption, not too unrealistic in situations such as mental testing, it is 
possible to select optimal sets of items without going through all possible 
combinations. A method proposed by Elfving to accomplish this is analyzed 

: and then demonstrated through the use of two illustrations. The common 
and often used procedure of observing the magnitude of the correlation 
coefficient as an index in item selection is shown to have some merit in the 


* single-factor case. 


The question of type and number of items to be used in a test of mental 
ability, an attitude scale, a personality inventory, or biographical inventory 
is а familiar topie in psychological testing. This paper will attempt some 
resolution of the problem for the restricted situation where the item variables 
obey a known factor structure. For simplicity the generic term zest will be 
taken to represent a collection of items whether dealing with attitude, 
personality, etc. The classical situation where test response is a linear func- 
tion of the item responses will be considered. This, of course, may be some- 
what unrealistic in some situations, e.g., biographical inventories may call 
for nonclassical approaches in item selection. On the other hand, even for 
what may be termed the classical situation the question has not been resolved. 

Assume that the test response is to be used as a predictor of a criterion 
variable. The number and type of items selected for a test, therefore, are 
governed by the usefulness of the resulting test response for prediction 
purposes. In the usual techniques of item selection, the size of a correlation 
coefficient between an item response and the criterion indicates how well the 
item aids in the prediction made by the test. While the correlation coefficient 


appears to have pragmatic value for tests of mental ability, it falls short in 
2: p should be remembered that the 


a number of other testing situations. It 
] tool borrowed from those who 


correlation coefficient is a methodologica ‹ r : 
conceptualized its use in anthropometric settings; it may not be efficient 
in all psychometrie situations. А conceptualization of the problem which 


*Work performed under contract AF 41(057)-244 with the School of Aviation Medi- 


cine, Randolph AFB, Texas. à 
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11,12, --- , y ; 2 are assumed to be random variables having joint distribu- 


Е are, respective] 
It can easily be demonstrated [1] that the bes 4 a RN 


б. ELFVING, В. SITGREAVES, AND н. SOLOMON 191 


composed of the same common factors and differ at most in factor loadings 
and specific factors. Thus, 


Q) v; = Quy + s Фань +в  (-—1,-,N), 
@ 2= о dn +m 
where 


(2) the loadings a;; , c; are known constants; 

(ii) the (unobservable) specific factors € , +-+, еу, 7 are random variables 
with mean zero, distributed independently of the unobservable (latent) 
common factors Yı , +++ , ys ; 

(iii) the e; have known covariance matrix (in most of what follows, this 
will be assumed to be diagonal); 

(iv) has variance o? and is uncorrelated with the e; . 

For the common factors у; , consider two different models which, how- 
ever, lead essentially to the same selection technique; 

(о a) fixed constants model where the у; are unknown constants; 

(v b) random factor model where the y; are random variables, with 
mean zero and a known nonsingular covariance matrix T. 

In model (v a) the y; may be thought of as factor values pertaining to the 
particular individual for which z has to be predicted. In (v b) the individual 
is thought of as belonging to a population with known characteristics. 

In predicting z, only linear unbiased minimum-variance predictors will 
be considered; i.e., for any selected set of n items out of N items, a predictor, 


is taken to be the linear combination 

(3) е > 9:2, 

which satisfies both E(4 — 2) = 0, and E(* — z) = minimum. This is а 
reasonable criterion for estimation quite often used in classical multivariate 
analysis. Suppose for the selected set of n items in matrix notation, 


(4) x= Ayte, 
(5) 2 = су + т, 
апа 


(6) E = cove = Ee’). 


Then, in the fixed constants model, the best predictor of 2 on the b: 
* g=c(A'n A) АТТ, 


asis of x is 


(7) 


with prediction variance 
2 


(8) Е — 2)? = 22-47% +o. 
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In the random factor model, the best predictor of 2, on the basis of z, is 
(9) $ = c'(T^* + А’ AY? A'S, 
with prediction variance 
(10) EG — 2) = (T + AEA +o? 
"These results are merely stated since they are well known in multivariate 
analysis. 

The Selection Region 


Now consider the selection problem. The problem is to choose n out of 


N items. There are many possibilities but that set of n items is desired which 
will minimize the prediction variance Е(2 — zy. Naturally a procedure is 
preferred which makes it unnecessary to look at all possible sets of n items 
to arrive at the proper choice. Considerations of all possibilities can be done 
for small п and small № , but even when № = 10 and n = 2, it is quite tedious. 

The prediction variance is given, for the two models considered, by 
(8) and (10). Since c? does not depend o 


n the selection, the problem reduces 
to the minimization of 
(11) У = c'M^t, 
where 
(12) 


M = M, + A'24 


Rr 
with M, — 0 in the parametric model, and M. о = Т 
model. The variable elements 
т X n matrix У, both of which depend on the choice of items, 
From now on, assume that the specific factors are uncorrelated, with 
variances сї. That is, for a selected set of n items, 


o 
in the random factor 
are, of course, in the n X k matrix A, and the 


Ў 2a % апага 222 ў алал, 
i=l i e 
n 


с ісі 6; fet сі 
2 

(13) АЗСА = Sada у ба ааа 
LI isl OY ісі 0; 

= 2 

ааа b» аа а 

? === 
ізі Cj ісі 6; С 


i=] 0; 
Denote by а; row 


tof A written аз a column vector, Then, 
n н 
(14) Аза „ан ч 


, Р а; 
Gt gi £a, where и, = % 


с; 
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that is, 

а ала; ааа 

с: о о 
(15) шыш = биде Bii as йй 

gi с; c; 

2 

адат Qizlik Qi: 

E] 3 7% 

6; 0% Ci 


is, for every i, а k X Е matrix of rank 1. The sum Ут. нш) has for elements 
the moments of the k-dimensional item population consisting of the points 
Uy, *** Un , each with weight 1. 
Returning to the original numbering of the items, i = 1, «+: , №, and 
denoting by о the selected set of n subscripts, (12) may now be written 


(16) М =М„+ L uui. 


The k X k matrix M is the information matrix of the experiment, i.e., of the 
| represents the information 


selected set of observations. The sum Duu; 
offered by the items in w. In the random factor model, M, = T^ may be 
said to represent the a priori information contained in the assumption that 
the factors are random variables with zero means and covariance matrix T. 
Jt may be noted, incidentally, that the constant term Мо in (16) can also 
be used to take care of any fixed. source of information, such as possible 
“compulsory” items. 
It is seen that the effect of а particular item-number 2, say, depends 
solely on the vector М, ie. on the reduced loading vector obtained by 
standardizing the item variable to have variance equal to one. In the fixed 
constants case, one may multiply the и; by а common factor without affecting 
the minimization problem at hand. As а consequence, the item variables 
may actually be standardized to any common variance, not necessarily 
unity. 
‘Also note that two items, one with vector u; and the other with vector 


— u; , yield the same contribution to the information matrix. For this reason 
and reasons of symmetry which will become clearer later on, each item will 
usually be described by means of the pair of opposite points + иг. These 
points, in k-space, will be referred to as #ет points. | | 

It is natural to think of the selected set w of item points as occupying à 
certain "selection region" S which will, of course, depend in some way on the 


totality of available item points. Before attacking the question of how to 


find this region, it may be useful to discuss briefly the cases k = 1 and k = 2. 
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When Ё = 1, all symbols in (11) and (16) denote scalars, and the problem 
reduces to minimizing 


с? 
SS -- зек 
mo + Ж и; 

u 


by a proper choice of w. Thus one has simply to select th 
| u; |. If the z; and u; ‚ аз often in fac 
one, a; + о? = 1, and hence 13 = а? 
| u; | are then the same as those wi 
loadings with respect to the single 


(17) F= 


en items with largest 
tor analysis, are standardized to variance 
[04 = ai/(1— аў). The items with largest 
th largest | а, |, i.e., those having largest 
common factor. In this case, however, 
(18) а. DN 
La, m бг?” 


50 that this procedure is equivalent to pickin 


5 the x, having the highest 
absolute correlation with у, 


- This procedure is usually follow 


In the case of = 2, it is р 
to lie precisely оп the 
then 


(19) Ti = any, + 220 + е 


= e (суу, F са) ta , 
and 2/c;À, provides an unbiased estim. 
is of/(o4,)? = 1/M. Accordingly, it 


metrically with Tespect to the line c, For example, suppose 


uw = ( М, + ke. Aes ) 
(20) VE ee Vitam), 


re A 
ш = zs + kc. = tc, |. 
à (2 TELA ed" s) 
Then 


(21) (в 2) \ Wife 
2 Bh m va pg © + №) +5 аа), 


61 б 
апа 
(22) мо + с (8 К =.) 

2% 71 о, 
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is an unbiased estimate of c'y with variance 


(23) (сі + с), 


Since А is the distance from the origin to the point of intersection of the 
c-vector and the line joining и and uz , it is clear again that the two item 
points, after the elimination of the orthogonal component, provide an esti- 
mator of с’у which is better the farther off in the + c-direction they lie. 
These heuristic remarks make it plausible that the selection region S 
will have to comprise the outer parts of k-space with regard to the directions 
= сапа will have to fulfill the additional requirement that the item points 
in S should in some way balance each other with respect to that direction. 
It turns out [1, 2, 5, 6] that S may be taken to consist of two symmetrical 
half-spaces (a twin half-space) bounded by two parallel planes Йи = = h; 
the direction £ of their common normal will depend in a certain manner on 
the item points included in the selection region, the latter becoming in this 
way implicitly determined. 
In order to formulate th 


а continuation device which will lea 


will greatly facilitate the solution of our problem with 
ing the practical application. For this purpose consider, instead of the previous 


M defined by (16), the generalized information matrix 


N 
(24) М= М№+ >; puut, 


py) is subject to the restrictions 


e result just sketched, it is necessary to introduce 
d to a simple optimizing criterion. This 
vithout essentially affect- 


where the allocation vector р = (рі 


N 
0 pd SN. wae iom 
4) and (25) contains the set of 


tems for which p; — 1 will be 
= 0, as nonselected, and 


(25) 


ces given by (2 


Obviously, the set of matri ^ 
lar p, those 1 


matrices (16). For any particu eig 
referred to as totally selected, those for Td A 
tho i = as fractionally selected. i - 
p bes сва | fractional p, (in the parametric 
Case) may be насен Imagine for à moment pnl ни А 
тау be independently repeated, an of Migros mos >i a prone 
zati is allowed. 16 И, ҮН Susy 
I 5 о ын at the different gene are re- 
peated. The ане у matrix of the resulting experiment may be written 
(26) M = naa 
where the p; = ni/r Vary from 0 to 1 p ш 
condition Хр; = n. Since the factor 7 


= гри, 
multiples of 1/r, subject to ће 
viously irrelevant to the 
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minimization of (11), for large r, the problem is essentially a modified selection 
version of that formulated above. In particular, if n = 1, and r is large, one 
is concerned with the allocation problem treated by Elfving in [3]; the earlier 
problem thus appears a special case of the present one. 


After these preparations, one may state the following propositions 
B, 5]. 


THEOREM 1. The scalar V. ; defined by 
N 
(27) V-—cM"e, where M= Mo + © puu! 
ізі 
as a function of the vector P, 


the allocation vector р to yield 
for a certain number h > 0, 


has a minimum on the 


domain (25). In order for 
this minimum, it is ne 


cessary and sufficient that, 


І whenever | "М-ы, |>h 
(28) Pi = 
0 whenever (e Mu <h. 
Moreover, there exists alwa 
ponents. 
Recall that M is a k X k matrix, hence eM 
vector, and c'M-'u, is a linear form in the 
content of the theorem is that the select 


ys a minimizing р with at most k fractional com- 


P: (when they are taken to be as fey 
will be from n — k 


of the fractional 


P: to unity, i.e, 
items on an equal basis with the re. 


attained by any total selection of n і 
p will contain only 178 
18 provided, 

For a proof of Theorem 1 in 


ў the parametric сазе (the random factor case 
goes quite similarly) the reader 


is referred to 


but no meth 
graphic pictu Points may lead to a 2004 guess, Е 
situations, а method ha, 
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(i) Find the matrix A with elements 
X 
(29) Aa = У usua (Gh = 1s, k). 
ізі 


(ii) Find the vector y = A''c, i.e., solve the equations 


мт Hott + МЕ та, 


(30) Ұз 
Nati bos + Nae = 6 
(iii) Find, for each 2, the quantity 
(31) w; = у: = У тш: 


and select the n items with largest | w; |. 
The selection found in this way should provide a good first guess and, of 


course, may be checked by means of Theorem 1. If the criterion is not fulfilled, 
one may try to improve the solution by exchanging one or more of the selected 
item points for others in the neighborhood of the boundary planes. An 


artificial example of such a procedure is given in a later section. 


=2 


A Realistic Example when k 
To illustrate the meaning and use of Theorem 1, two examples will be 


considered, both of which involve à two-factor structure and assume а fixed 
constants model. The first example is based on data made available by the 
Educational Testing Service. These data resulted from responses to six 
items used to measure aptitude for success in law school and а criterion 
variable which measured success in law school. A factor analysis of the six 
items and the criterion variable resulted in two common factors plus specific 


factors. For this illustration, 


(32) a, = aly + 6 6-1,2, 5-2; 
z-cydtm 
Where 
af = (848 228, а = (898 195), as = (840 .122), 
— 170), а = (869 .204), 


@3) af = (481 —.216), a= (647 
c = (641 — 2205). 
в j 7 are assumed to be in- 


" ина 59 ; 
ше asi ha ; o. The variances 


The 
(unobservable) spec cables with mean zer 


dependently distributed random У 


of €1, +++, €; аге d M" 
(34) of = 280, а” 268, E = 5" 
с? = .722, вй =з 500; баз е 
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The vector of (unobservable) common factors У = (01у) is assumed to 
be a vector of unknown constants. 


Then, if u; = а/с; , 


Qs) = (707.469, ш = (1.608 376), ш = (1.588 931), 


mi = (560 — 254), ш = (.871 —.231), ш = (1.927 452); 


and 
шщ = | 3.122 27] шш = [ 2.586 x 
.829 220], 605 141], 
шщ = [ 2.522 | шш 11 1320 кі 
(36) 367 053]? —444 065], 
"Mw 759 ж ке ыб EL 
—.201 053], S71 — 204 


The item points u, eos 
Figure 1. 

For a given set of 
basis of х is 


(37) 


` з № , and the vector с are shown graphically in 


26, BAY Li, peee » Ti, , the best predictor of z on the 
z = cM, U,s*, 
where w denotes the selected set, of subscripts, and 


М. = Хиш; 0, = (ш, ш, o... к), а 2Xn matrix; 
9$, 0% Tin 
The variance of the estimate is 
(38) Ee — 4? = ом, +. 
М. may 


be considered the information matrix for the Selected set of obser- 
vations. 


Consider the generalized information matrix 


в 
М, = я put , 
ізі 


оп vector p' = (p, 


(39) 


where the allocati 
tions 


(40) 5 


Pz Po) is subject to the restric- 


That is, two out of the six items are desi 


red. Then 
(41) 


„————. 
- — у, 
A 
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as a function of the vector p, has a minimum on the domain (40). In order 
for the allocation vector p = p* to yield this minimum, it is necessary and 


sufficient that for a certain number h > 0 
T 1 whenever |c’M>-u;| >h 
0 whenever |o Мәш] < h, 


where p* is the minimizing vector. That is, the wholly selected item points 
lie outside of two parallel lines which are symmetric with respect to the origin, 
while the totally nonselected item points lie between these lines. (Totally 
selected and totally nonselected item points may also lie on the boundaries, 
i.e., parallel lines.) Any fractionally selected item, say n; , must be on one of 
the lines defined by | cMziu; | = h. At most, two items may need to be 
fractionally selected. 

Suppose in the present exampl 
to picking the best two wholly se 


e, where n = 2, we first limit ourselves 


lected items; that is, the possibility of 


fractionally selected items is not permitted. If one examines the 15 possible 
combinations of the two items in pairs, it is seen that items 4 and 5 are best, 
i.e, they yield the estimate with the smallest, variance. Writing ро = 


(0, 0, 0, 1, 1, 0), the variance of this estimate is 
(42) Мс = 381. 


is the minimizing vector when 


Theor hat if, in fact, Po 
heorem 1 states that и, e is à number һ such that 


fractionally selected items are admitted, ther 
(43) “Муш | > h, | Муиз | > № [омуш < 6-1,2, 3,0) 
However, 

1.043, М | = .950, [Мэ | = .939, 


337, | eM sits | = .517, 
and it will be necessary to 


ау leM | = 
| eM | = 

It follows that po is not the minimizing vector, 
consider fractionally selected items. | 

Since i d 5 are the best pair € € 
based eig rie E d 6, for example, 18 447.01), it m! hrs o 
these items be included when fractional selection i love * ge ie say 
indicates that there must be 8 pair of parallel lines such that the 

i i ine. { 

ally aa % an if the two parallel lines are qua аа 
items 3 es i item 5 lies outside the lines and items 1; аш Ly etween 
them. Thus on is led to attempt 8 fractional solution of tà 


р* = 0 от 1-71 0). 


| "Мм, | = 1-141. 


of items (the variance of the estimate 


(45) 
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Determine 7 such that 
E'M iüs: = Муи, . 
"That is, 


[.641 Br — 012r 345 — c br 
345 — БШ 1.079 + 2.202; ||. .231 


= [64 Bes 7 8 1012" .345 — a .566 
345 — .511г 1.079 + 2.202r | —.254 | 


(46) 


From this equation r = .019. 
It follows that 


(47) р* = (0 0 019 .981 1 0); 

M, = | 1.1208 BE]. 
Ln | ҖЕ 1182]? 
(49) има = (3506 —.7399; 


І 


273,  c'M;u, = 286, c'M,u, = .386, 
сМ = 341. 


Му 
сМ, = 38, cM 5s = .476, 
Thus, the conditions of the theorem are satisfied, and p* is the minimizing 
vector. The variance of the associated estimate is .376. When this is con- 
trasted with .381, the variance when item 4 and item 5 are used totally, 
2 slight improvement by the use of fractional allocation 1s seen. 
It should be remembered, however, that the modification of the problem 
by admitting fractional allocation is an ad hoc device, applying primarily 
k; its justification is to be found in the 


When n is considerably larger than t x 
paragraph following Theorem 1. Thus, in the present example, the practica. 


problem is to find the best integral solution, i.e., the best pair of items. The 
fractional solution (47) suggests that items 4 and 5 comprise the best choice, 
and this is actually the case, as has been checked by comparing the variances 


corresponding to all possible choices of two items. 
An Artificial Example when k = 2 | 
For the selection of two out of six items, in the ргеуіойв example the 


variance of z actually was computed for all 15 pairs in the two-factor structure; 


ir ractional allocation. 
it w. improve on the best pair by fractional : } 
dnas ad e bined with graphical considerations, could 


It was show 2 rem 1, com : ! 
di агана Be paie of items without gomg through all possible 
pairs This becomes especially important when М and л both increase. and 
thus rate out the examination of all possibilities. It also becomes important 
when £k, the number of common factors, increases. 


(50) 
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The application of the theorem in a more complex situation will be 
demonstrated by the following artificial example. The problem is to obtain 
from the 10 items, first the optimal sets of 4 items, then of 5 items, and 
finally of 6 items. А common factor space of two dimensions is assumed. Thus, 


ж; = ау + ei @=1,...,10, 


(51) 
2 = су +n, 
where c' = (2 1). Assume 
ш-(4 0, w= 3), 
w=(-2 0), ш=@ 3), 
(52) w=( 3 1, w= 3), 
uie x 2), шш (6: М), 
w=( 4 2), щ=0 5); 


= 


wui = 


о оо 
2 

ДЕ 

І 

Lat 


| 


St а р 
E 
ES 
І 


(53) шш = 


щщ = 


шщ = [ 16 8 


вю нон 


шш = | 25 All 
20 161” 


uso = | 4 ш], 
10 25 


plex situations, presented earlier, 


3 
ES 
5. 
І 
Lis 2 
| 
we ко ПЕ GS 
pu 


(92) 
ны 
Е 


Applying the approximate method for com 


10 120 | 
54 = ши = : 
ы E х | 67 88 
Then 

ЕЛ 78 = 

д 
ii —68 120 
and 
a —16). 
t mg ~ (88 

ы : ed with proportionality factors. 


not be concern 


tted graphically Plotting the direc- 
e 3 


In (55) and (56) we need мы 2e 


The item points are plo 
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i in Fi 2, and imagining a straight line perpendicular to it moving 
аг then the maa in which the item points are swept over by 
this line indicates their order of preference in the first approximation. Ба 

lin = 4, 1.е., 4 items are to be selected, one is led to the selection 44 
1, 5, 8, 9. Forming the corresponding product sums, 


-46 
(57) My ~ | ын 


| ‚ Xt Q2 р). 
—46 93 


In the direction of Yı , the four outermost item 
5, 8, 9. The theorem shows, then, that choice I i 

Next, take n = 5. Usin 
y-direction, one tries the set T 1:1 


points are still numbers 1, 


е outermost points 1, 3, 5, 
makes it plausible that a S 
ting fractional selection) m: 
troducing unknown weights 


8, 9, i.e., the set first 
election of form II, : 
ight give an optimal 
Pa , p; With sum one, 
(58) М, ~ E + 9р. + 9p, 46 + 3p, + ж. 

46 + 3p, + 9p. 29 + р; + 9p. 


points 3, 7 to lie on the bound 
М7% must have direction (1, 


Іп order for the 
the vector y = 


(59) 


ary line of the selection region, 
0), which gives the condition 
—2(46 + 3p, + 9р) + (93 + 9p, + 9p.) = 0; 

hence (noting that Pa + р; = 1), р, = 2/3, р, = 1 
allocation vector is p* = (1,0, 2/3, 0, 1, 0, 1/3, 1,1,0 
make a choice between selections П, and IL, . The 
V = t M'e are (note that the factors 1/|M | car 


/3. Thus the optimum 
). In practice, one must 
corresponding variances 
nnot be omitted at this point) 
y, = 302) — 2(49)(2)(1) + 1021) 9% 

= 


102(30) — 49019) = бб = 0.03945, 
= 38@) — 2(55)(2)(1) + 102(1*) С” 
"PS 102(38) — 5555) = 851 = 0.03995. 


line, item 7 has wi 


: the boundary 
eight 1, and item 2 has weight 0. 
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Early in the paper it was stated that the optimal (n 4- 1) set of items 
need not completely contain the optimal set of n items. However in many 
practical situations this will occur and the present exercise demonstrates 


this event. 
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A METRIC AND AN ORDERING ON SETS 


FRANK RESTLE 


MICHIGAN STATE UNIVERSITY 


Beginning with sets of arbitrary elements, concepts of distance and 
betweenness of sets are defined. Since betweenness as defined is not transi- 
tive, an investigation is made of the conditions which ensure desirable 
regularity. It is found that a straight line or linear array of sets is a generali- 
zation of nested sets (Guttman scales). Close relationships among the notions 
of distance, betweenness, and linear arrays are demonstrated. Parallel and 
perpendicular arrays, dimensions, and multidimensional spaces are char- 


ncterized. 


Basic to many psychological discussions is the concept of similarity, 
which is used to arrange objects or events. Two things which are quite 
similar are psychologically close together, and two things which are quite 
dissimilar are psychologically distant. This interpretation of dissimilarity as 

ive a simplifying, quantitative 


distance is the basis of numerous attempts to gr І 
sychophysies, learning, and other fields. 


structure to psychological data in p 

In contrast to these metric, quantitative developments are more 
qualitative discussions of similarity and the arrangement of objects, using 
the idea of common elements to account for similarity. In learning, such 
views are common in current mathematical theories [3]. Some recent efforts 
have been made to generate а metric analysis from а. ims sar 23 
siderations of learning theory [2, 3] and psychophysies [4]. Ӯ ds 
shows how a distance and ordering can be defined on sets, and generates the 


begumings ef the resulting geom 1o individual locations or 


itr: considered have 1 
Tis ae ee р to the sets of which they are members. 


arrangement except with regard se is t i 

Thus the eae is entirely qualitative. The purpose 4 D hd 
what internal evidence would justify saying that a к | apparatus 
and to construe a concept of distance ресе ra 


used is elementary set theory. " ык 
The following mathematical concepts and notations 


the universe. 


heen B is the set containing ev 


aL 

4 U erything in either А 
M union or set sum; А 

(e 


or B or both. К ете ыы 
intersection or set produc 
п on to both А and В. 
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4A 1 Bis the set con 
m 


elements comm 
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set-complement; А is the set containing all elements of 41 which are 
not in A. 
(= the relation of inclusion; А С B if and 
also members of В. 
m а measure function; if S* is a set of sets (5: у. Sx, 
measure function, then the following axioms hold. 
M1: for all S; , m(S;) > 0. 
М2: m($) = 0. 


M3:it 8; AO 8; = 8, then m(S; U 8,) = m(S;) + m(S;). 
{a, b} is the set whose only members are a and b. In general, entities enclosed 
in curved brackets are the members of a set. 
(a, b) the ordered set whose first member is a a 
general, entities enclosed in angle bracket; 
cated order, of an ordered set. Two order 
they have the same elements in the same 


identical if they have the same members. 


Consider a universe q of arbitrary element, 
аз the universe of stimulus elements, 
models, or which might also be the s 
population of subjects, etc. 

Certain subsets of a 


only if all members of А are 


sj and m is a 


nd second member is b. In 
8 are members, іп the indi- 
ed sets are identical only if 
order, (Unordered sets are 
—order is immaterial.) 

8, which may be interpreted 
cues, etc., in learning or perceptual 


: et of responses to test items, or the 
; In other applications, 


sider a set s = {б Ва. 
set, which may be the number of elements in 
receive different weights) the sum of weights 
more abstraet idea of magnitude. The funct 


by the operations of union, inters 
countable number of times. Т, 
will be defined on all the sets 


Metric of Dissimilarity 


Let the dissimilarity or dista 


nce between two sets S; an 
Di; . For D to be a metrie dist 


t 5 d S; be Written 
ance, 16 must, satisfy the f, 


н ollowing metric 
axioms. 

Ахтом 1. Di; =0 ifand only if i = j, 

Ахтом 2, Dy > 0. 

Axiom 3. D;; = Dya 

Axiom 4, Di + Dj, > 5n. 


Within the present system, the distance D., must be деб; i 
І stem, T ned in te 
sets S; and S, , combinations produceg by intersectio Үк ыы 


n, union, and comple- 
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mentation, and the measure function m. The nature of the elements in aL 
does not in general give a concept of distance between elements upon which 
to build the concept of distance between sets. 

Other things equal, the degree of dissimilarity between two sets depends 
on the measure of noncommon elements, 1.е., the symmetric set difference. 
The first proposal of this study is to let the measure of the symmetrie set 
difference be the distance metric. 

DEFINITION 1. D;; = т [(S; V $;) М (S: O 8;)]. 


The expression on the right is, of course, the symmetric set difference. 


THEOREM 1. D;; is a metric. 

Proor. 8; U 5, = S, OÑ В; = S; , whence Da = m(S; Г Sj) 
= т ($) = 0. If т vanishes only for the empty set, D satisfies Axiom 1 ofa 
metric function. Axiom 2 is satisfied because measure functions have non- 
negative values, and Axiom 3 is satisfied because of the symmetry of the 
expression in Definition 1. Axiom 4, the triangle inequality, is shown to 
hold by partitioning 4L into the following eight cells (see Figure 1): 


S;,AS; S = А, ; BA Geli = a 


8,08; S = Ау Se OS) OS = 44; 
SAS, A& = Аз; SOS; O08 = А; 
BAS AS = А; SOAS: OS = Аз. 


Figure 1 
with the cells of t 


ivi u, Iti artition 
Three sets, Si , 8; " and Sr , іп a universe “s he resulting p: 
, 
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For compactness in what follows, let т(А;) = a; . Now 


D 


и = а фа. + as + а, 
Di = а a; + as + а, 
Di 


І 


аз Fa caca; 


whence 


Di + Di = Di + 2a; + 2a, , 
and 


Da t Di > Dy. Q.E.D. 
An Ordering on Sets 


say that S; is between 8; and S, . Two 
such a description; first that S; and 8, 
re not also in 5, , and second, that S; 
i nor S, . Figure 2a shows 

ows an 5) which 
second condition. 
This concept can be written as the following definition, 


Derinition 2. S; is between S; and S, (written bii) if and only if 


OSAS AS =з (Аш Figure 1 is empty), 
(ii) 8; 0 8; 8, = ø (A, in Figure 1 is empty). 
A major justification of definit; 


ions 1 (the metric) and 2 (betw 
found in the following theorem, 


eenness) is 


THEOREM 2. 1) bise then D;; + Dj, = Di, ; if т vanishes only for the 
emply set then the converse is true. 


Pnoor. From the proof of Theorem 1, 
Di $ Dj, = Di, + 2a, + 2a, 


= Da + 2т(8, Гу S; S) + 2т(5; (Y 8, A &). 


which fails to meet Ше. 
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FIGURE 2 
An example of ordered sets (а) and two examples of unord 


ered sets (b) and (c) 


By hypothesis, b;;, , the sets S; O S; A 8, and S, N 8; AO 8, are empty, 
whence D;; + Dj; = Da. The converse is obvious if аз = @ = 0 implies 
that A, and А are empty. Q.E.D. 

Thus, b;,, means that, in an abstract sense, S; lies on a straight line 
between S, and S; , in that a path from 5; to 5, through S; is just as short 
as any path. A preliminary investigation has shown, however, that one does 
not easily build a rigid metric space from the relation b. The main trouble is 
that b is not transitive in what would be a most useful way. The problems 
are shown in the two counter examples proving Theorem 3. 


TuzonEM 3. It is not the case that biin and б, implies bis. 


Proor. Two counter examples are given because of their intrinsic 
interest. 
First counter example: let S; — {1, 2,3}, 8; = {2,3,4}, 8. = (8, 4, 5}, 
and S, = {4, 5, 1}. Then b; and ба, but not biim because S; O S; O Sn = 
{1}, whereas it must be empty for biim tO hold. 
Second counter example: let 5; = {1,2}, Si = {2, 3}, 8, = B, 4} and 
S, = {4, 5}. Then bij and bjzm , but not biim because S, N 8; O S, = 18), 
whereas it must be empty for biim 10 hold. Q.E.D. 1 
These two examples correspond to what may be considered psychological 
reality. The first counter example shows а case of looping in which a series of 
neighboring sets, eac hbors, may lead around and back 


h between its two neig s 
to the starting set. This is the apparent structure of the color circle, the 
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array of hues from red to yellow to green to blue to purple and back to red. 
In the second counter example, one detours from S; to S, through some sets 
which may be quite unlike either. For example 
be made, varying from starkly si 7 
middle of the series could have 
ness, shared by neither extreme. 

Since a long straight line, or linear array, 
from shorter segments in a reliable way, 
array let us seek a reasonable general di 
which loops and detours can be avoided. 


of sets cannot be built up 
for use as а definition of a linear 
escription of the conditions under 


The Special Сазе of Nested Sets—The Guliman Scale 


Consider a finite Sequence of sets S, , 5, jos 


fort = 1,9, ... › ^. These sets are nested withi 
the largest. Let such a Sequence be called М. 


5: Sa such that SC S 
n one another with S, being 


рер 
О 


PS 


Б 
> 


Be] 


5 
> 


Si 
5 


Po 
> 
р] 

42 


А 
74747 02454 
(5502456 


Ficure 3 
А nested sequence of Sets as referred to in "Theorem 4 
THEOREM 4, IfS8;,8 


i, and S, are in № andi < j < k, then Usos. 


1 N S, Ба subset of S; ГҮ 8, whisk i 
and S; N S; N S, is а subset of 5; O S, whic Mert 


h is empty, establishing b;;, . 
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COROLLARY. If 8; , S; , and S, are in N and i € j € k, then D;; + Dy, = 
Dj. 

At least up to this point, the material given is familiar and found in 
standard mathematical texts (e.g., [7], pp. 16, 24, 107, Exercise g). 

'The condition that an array of sets be nested is sufficient to ensure 
transitivity of betweenness and additivity of distances, but it seems quite 
restrictive in that it certainly does not correspond to qualitative scales. 
However, it is possible to make a qualitative or substitutive scale from two 
nested arrays and a constant set. 


А Linear Array of Sets 

The following development, like that above, will be restricted to finite 
arrays of sets. No great difficulties are foreseen in generalizing, but the 
ability to deal with finite numbers of sets is an advantage in empirical appli- 


cations of the system. 

DzrixrrION 3. Let А* = (45, А», +++, An) and B* = (B1, B2, ++, B,) 
be two nested n-tuples of sets, with the restriction that A, O B, = ø (whence 
all the A, are disjunct from all the B;). Let C be any set in 8% such that 
A, OC = В, OC = ø. Then the n-tuple of sets 

Lt = (Га, De, = уа) 


where 


І; А; U Bain UC, 


І 


is а linear array of sets. 
Тнконем 5. If L: , L; , and Гл are in а linear array L* andi <j sk, 
then bij, . 
Pnoor. Partition L; into А; , B,-;4i , and C. 
By the hypothesis that A* is a nested sequence, and Theorem 4, 
А; ПА, ПА, = 9. 


Similarly, 
Bua n В.а Bra = 9, 


and certainly 
Cr 6n cC = 8. 


Now, 


L;A L; OL, = (A; U Boa i e o 
fA (A; U B,-;a U ©) ГҮ (А, УВ, ра „ C) 
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which, because the A's, B's and C are all disjoint, is equal to 


(A; A А, с A) U Bia б B. n f Bia) MJ (CAC A C) = $. 
Similar argument shows that 


BANDA D, = д; Q.E.D. 


Conouny. If L* ва linear array of seis and < j < k, then 


Dj TDi, = Dj, . 
Pnoor. Theorems 5 and 2. 


Note that nested sets are a s 
example, В; = ¢ for; = 192, 5%. 
of Theorem 5. 

The restrictions of a linear array 
sufficient to ensure transitivity of bet: 
now be shown to be a necessary condi 


pecial case of a linear array in which, for 
› п. Therefore, Theorem 4 is а special case 


me restrictions will 
tion for such transitivity, 


THEOREM 6. If R* = (R, , R, 9202 , Ra) ds a sequence of sets such that 
for alli, j, k, = 1, 2, , n, andi < Í S k, b; , then R* їз a lincar array 
of sets. 


Proor. It must be Shown that there exist nested Sequences A* 


(Ai, ++- , Aj) and В* = (Buy +=, В anda set С such that №, = A 
Bates U C, and A, ( B, Eu = бапа B,C = ø 
(i) Define, for an example, 


CI 


i 


сМ RAIN ace OR, 
А: = ROR, , 
B= М. 

Gi) Show that А, Cd 


(=R, iy T). 


әз. This is equivalent to the sta 


tement that 
(R: б Ry) © (Ris с RY 


or 

(Р, с R,) с (Risi б Ry) =й, 
or 

(R: A Ry) б (В. U Ry) = а, 
Multiplying, 


ШАРЛАРЫ ТАЛАР ТАН 
Which is true: the first 
obviously empty. Simil 


1) =, 
term is empty by b, 


+1 , and the second term j 
ar steps show that the à t r^ 


B's are пез ed 
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(iii) Showthat А; У Bnin UC=R;. 
A; U Bia U C = (В. №) U (Goss ТАР ЗАР (Ri A Be) 
= (R: O Ř,) U (R: OR) U (В, A Ra). 
Factoring the first two terms gives 
А; U Bain U О = [R; N (Ё, U R)] U @ ПЕ.) 
= [R; A R A R)I (В, OR). 


By the hypothesis D; ;, , it is seen that R, O R, = R; O (В, O R,). Substituting 
this in the above equation, 
А; V Bm V C = (ROR C R)] V R O ELA R,)] 
=R;. 
The rest of the proof is trivial. 

Theorem 6 shows, in a formal way, that any sequence of sets throughout 
which the relation of betweenness holds may be dissected into two nested 
sequences, running in opposite directions, and a constant remainder. This 
gives a sharp meaning to the concept of a substitutive scale, for as one moves 
from №, to Бұ, part of the elements of В, are removed, and some new 
elements from A, are substituted. There is no requirement that the measure 
added must equal the amount removed, though this may be an interesting 
special case. . 

From Theorem 6 it is seen that а linear array of sets is well represented 
by three generic sets, A*, B*, and C. A, and B, will be called the poles, the 
nested sequences А* and В* the polar arrays, and C the core, it being common 
to all elements in the original array. 


Multidimensional Arrays 


In current multidimensional se 


ment in Euclidean space. Attneave [1] and Lan ‹ 
Euclidean rules for computing the distances between objects which vary on 


more than one dimension, suggesting that psychological intuition does not 
force us to the Euclidean assumption. The present discussion suggests a 
generalization of the above system to more than one dimension, giving rise 
to a non-Euclidean arrangement like those mentioned above. 


Derinition 4. Two linear arrays of sets, L* and 1/%, are said to be 
parallel if and only if their polar arrays are identical. 


Lemma 1. If L* and L™ are parallel arrays, Dy; = Ёш. 
from the fact that distance depends only 
on elements). 


aling it is common to assume an arrange- 
dahl [6] have suggested non- 


Proor. This follows at once 
on the polar sets, not at all on the core (or comm 
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Тамма 2. If L* and 1/% are parallel linear arrays, 


Бі) = Di, = Di Es Рос, . 


Proor. Partition the universe U into the following cells: 


Q, = цп Сес"; 0. HLALUANENG: 
Q = LAU AEN с, 9$ -Ln^oLnené, 
0, Сед"; 0, = LALA GA &, 

= LALA a, 

The cells are clearly mutually exclusive. They are exhaustive because seven 

other possible cells are subsets of C A Г, ( N L1 (also empty), 

and the other two possible cells, І. LOG СҰС” and І. 0) LIN GCNG 

are identical with Q, ; being comprised 


Lj which are not in the cores. А diagr. 


am of the sets а 
shown in Figure 4, Here, D;; = m(Q;) + m (Qs), Do, = 


Ш 


Fraung 4 
г and L/ and the partition referred to in Lemma 9 


The sets 7, 


and as stated by the lemma, Dip = m(Q, 
Di; + Doo. Q.E.D, 
In what follow. 


„+ т(0,) + т(0,) + т(0,) = 


5, а set of parallel line 


ar arrays will be referred to asa 
dimension, using the following notation. A dimension Xt isa Set of linear 
2) 1 i 

arrays, ГЖ) ж || ‚ LÀ, which are parallel to on 


е another, 
DEFINITION 5. If Xf = peo 


(1) 
of the arrays, C у 
qu. egy Leo is 


m жо. 5 Е 

= Же TORE a dimension and the cores 
ес form а linear array, then the set of Members of 
called a linear two-space, 
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Turorem 7. If S* is a linear two-space made up of the linear arrays 


а үт а) a) 
DAP = (DIY +++ EDS os Ду) 


(e w w 
L*) = (О, LV Ee) 


then the sequence of sets composed of the rth member of each array, i.e., LP? +++, 
(i) . . 

LY, «++ , Lt? is also a linear array. 

-space the cores form a linear array, and 


Pnoor. By definition of a two ores 
L® (^ С", the result is still a linear 


by adding to each core the constant set 
array. Q.E.D. 

Theorem 7 shows that there is no asymmetry between the two dimensions 
of a two-space. If X* and Y* are the two dimensions, the cores of X* give the 
polar arrays of Y*, and the cores of Y* give the polar arrays of X*. 

Further dimensions may be constructed by considering the joint core of 
a two-space, i.e., the intersection of all sets in the space. If one were to con- 
struct a sequence of parallel two-spaces; identical except for their joint cores, 
and if the resulting sequence of joint cores were a linear array, then one would 
have a three-space. This process can be repeated indefinitely, so that an 


n-space is defined for any л. 
and Description of the Space 


An advantage of Euclidean space, which has contributed to its use in 
psychological measurement, is the Pythagorean theorem, that the distance 
between two points which are apart, on various dimensions, by amounts 
Ti, ta, ++: а, , is given by d = "c If two sets in dicm of the 
present type are apart, on the various dimensions, by amounts 81,22, *** yay 
then the distance between them is d = 22»: : This is shown informally by 
applying Lemmas 1 and 2. The only differential elements must, by definition 
of an n-space, be elements in the linear arrays m the п aa Since 
the poles of the various dimensions are discrete, the measures are all added. 
That the result is unique follows from Lemma 2 and the symmetry of di- 


mensions, Я 1 | 
One gets an idea of the space by considering à small px eres 
Scan jn jus Eudldeen mapping, libe fif In PARES DIE LN 
represented b ints, and the paths from рош mee . 
Тһе Каи i ths from a to b in апу two-space 18 easily computed. 
Е о d by s sets in the other, 


If a and b are separated by 7 sets in one dimension an 
$). Of course, the steps need not be all of the 


the number of paths is d М 
same distance, but the total lengths of all paths will 
f parallel arrays, ап 


Along with the concepts O Е 
(in which the poles have no overlap), one can also con 


А Non-Pythagorcan Theorem, 


be the same. 
d perpendicular arrays 
ider oblique arrays 
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(in which there is some overlap between the poles). In Figure 5, with the 


о b by changes in 
n be produced ex- 
eption experiment) 
to b by successive 


Ficure 5 


А two-space with 3 


X 4 sets, each mapped on a point 
(The ten ways of 4 У 


going from a to b аге indicated.) 


Dij; = 


- 
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M m(A) + т(В) — 2m(A N B) 
n m(A) + m(B) — m(A N B) Е 


This distance is restricted between zero and one and does not have the 
desirable property of additivity when sets are ordered. Bush and Mosteller 
[3] define a similarity index т as the ratio of the measure of the intersection 


to the measure of one of the sets. That is, 
ҠА to В =т(А (^ B)/m(A). 


ince it is not generally sym- 


'This index cannot be the basis of a distance 5 
directional 75, the result 


metric. If one used the mean or sum of the two 
would be similar to Galanter’s proposal. 


With respect to the ordering of sets, both 
have presented ideas like the present one, and the writer also previously 


proposed a similar ordering [8]. Burke distinguished between substitutive 
and nested orderings in essentially the way done here, but did not give a 


general treatment of substitutive orderings and ways of escaping the intran- 
consider an ordering of the 


sitivity of the local relationship. Hays [5] does not 

present type, and instead embeds the numerical distances in a Euclidean 
space. In the writer’s opinion, this loses part of the advantage of the set- 
theoretic formulation. 

There are occasions in psychological theory in which a dimensional or 
metric approach is entirely natural and valuable, as in dealing with loudness, 
pitch, or hue, in the diseussion of generalization gradients in learning, in 
treating polarized attitudes, and in certain applications of statistics. On the 
other hand, the categorical approach has & firmer logical foundation and 
the advantage that raw data in psychological experiments are usually cate- 

he gap between the two 


gorical. The present paper is an attempt to bridge t А 
approaches by showing а way to develop dimension from purely set-theoretic 


concepts. It is hoped that this may help in unifying the mathematical ap- 
proaches to psychological problems. і 
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MAXIMUM LIKELIHOOD ESTIMATES OF ITEM PARAMETERS 
USING THE LOGISTIC FUNCTION 


А. E. MAXWELL 


INSTITUTE OF PSYCHIATRY, LONDON UNIVERSITY 


ion is proposed as an alternative to the integrated 

rameters of test items. The logistic curve 

г finding maximum likelihood estimates 
esented. 


The logistic functi 3 
normal function when estimating ра: 


is described; an iterative method fo 1 u 
of its parameters is given, and an example of its use 1s pr 


Finney [9] pointed out that the problem of estimating the limina and 
precisions of test items, discussed earlier by Ferguson [8], was closely analogous 
to problems in toxicological experiments. In particular he showed that the 
methods of probit analysis could be applied with advantage to Ferguson’s 
data to obtain efficient estimates of the parameters in question. More recently 

hat in toxicological work a 


Berkson [2, 3] and Anscombe [1] have shown thé 
useful alternative to the probit model, in which an integrated normal response 


law is assumed, is one in which the logistic function is employed. They point 


out that while the shapes of these two types of response curve are very similar, 
ilable for the parameters of the logistic 


simple sufficient statistics are ava! 3 еде "considering th 
curve. А his property Birnbaum 95 whi ) е 
racted by this prop on and Wald theories of inference and 


application of the Neyman-Pears s 
Statistical decision making to problems of efficient design and use of tests 
d some advantages of adopting the logistic 

taining the probability 


s A ingle ability," demonstrate ei Me ntm 
netior ormal ogive, 

that an pier aer iei ep would je а specific response pattern to à 
k-item test. While Birnbaum's approach (also illustrated in [6] and [7]) should 
be viewed as a portent of the shape of future test theory, it 18 useful mean- 
while to look at the logistic model as an alternative to the come model for 
the straightforward estimation of the limina and precision of test items. 


oblem as Conventionall 
[11]) it has been assumed that the 


“rue” ability 2, опа test sampling 
st correctly is given by the ex- 


The Pr y Staled 


Conventionally (for sen see 
о 


probability P;, that an examinee ^, den 
ап ability z, will answer the item j of the 


pression 
(4721/61 


(1) Puis = № 
221 


gr du, 


^» 
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where a; and c; are constants for that item. When Р; 15 0.5, %, 18 wre 
a; so that, following psychophysical nomenclature, a; 15 the lumen о 
iom. that point on the z-seale below which 50 percent of the Meu onn > 
the item. The constant с; determines how well the Item discriminates be Wee 
individuals of high and low ability. The problem is one of finding efficien 

estimates, à; and 6; , of the population parameters a; and c; . For complete- 
ness special reference should be made here to Lord [11], where he points out 


that (1) can be obtained, if desired, from certain assumptions, prominent 
amongst which are 


(2) that there is a continuous varia 


ble z underlying the item, and 
(4) that x is normally distributed. 


Lord then shows that if the metric x is the co 
measuring a single ability, z, is inva 
18 administered, and that this invari 
distributed in the group tested. It should be noted that 
referred to here is not determined by a rank- 
chories (except in the special case where the common factor is norm 


distributed). It i nt structure analysis. 
The probit of the proportion Pi; is defined [9] as the value of Y satisfying 


mmon factor of a set of items, 
riant no matter what test of the ability 


ance holds whether or not t is normally 


the common factor 
one matrix of interitem tetra- 


ally 


1 (Y-5) dia 
(2) PA = vz | € / du, 
T J4—o 
where Y is the unit normal devi 


ate corresponding to P 
probit of P is then linearly 


{к Increased by 5. The 
related to x by the equation 


(3) Foo axis). 
and by the familiar methods of probit analysis the constants o; ànd c; can 
be estimated. 


are tested at each of k 


different values 
‚ К). In toxicological work т; 


would stand for 


dividuals falling 


"i , ога proportion p, = 
to have passed the jth item of t 


(4) Pi = 1+ exp (-а — Вх). 


” 
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'The logit of the proportion P; , generally denoted by L; , is defined as 
(5) L; = n (P;Q) = а + Вх; . 


Writing a + вх; = 8(x; — в), в, which is equal to — a/ corresponds to 
P = 0.5, so that — а/8 is the limen of the item. 

Гог each of (2) and (4) it is seen that а transformation of the dependent 
variable P can be made such that if the transformed value is plotted against 
х a linear relationship results, the slope and intercept of which give the 
parameters of the original function. 


The Shape of the Logistic F unction 

If тіп (4) ranges from — œ to + 9, P ranges from 0 to 
(4) with respect to x, 
(6) dP/dx = ВР — P). 
Hence dP/dx = 0, when P = 0 and 1, so that the logistic eurve has both a 
lower and an upper asymptote. 

Differentiating (6), 
(7) 4Р/ах’ = (1 — 2P), 


a point of inflection at — 0.5, halfway 
a of this point, as already noted, 


1. Differentiating 


which shows that the curve has 


between the two asymptotes. The absciss 
is — a/B, the limen of the ‘tem. From this description it is seen that the 


logistic curve resembles very closely the normal ogive. For this reason, it 
is, on purely empirical grounds, suitable for fitting to proportions of individuals 
of different levels of ability passing a test item [8]. 

а Estimates of the Logistic Parameters 


- Maximum Likelihoo 
is the proportion passing 


In the ith interval, the probability that p; 
the item is 
| 


n; * групі ті 
(8) Ts ! (n; = r;)! Р: Q; : 


The likelihood function over all intervals 18 


n; ! лч 
; Рг: 
(9) Пут 9! 

The terms in the logarithm of this function which involve the unknown 


parameters, a and В, are given by 
L == УВ іт; log. BR: + (п; x г) log. Q;}. ^ 


i 


(10) 
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В кере E t 
ituting i ; 4) and differentiating with respec 
ting in (10) the value of P; from ( 
ө 2 дек” 8 "m estimating equations for the two parameters are found to be 
Р . ` 


(11) Упр: = B) е), 
апа 
(12) b» тар; — P) - 0, 


respectively, where P; = [1 + exp (— & — ёх), Р, а апа B being estimates 
of the population parameters P; , а, and В. These equations, based on the 


sufficient statistics У) np: and У npa: › have been given by Berkson 
[2, 4], who also shows how to Solve them iteratively. 'The procedure 15 as 
follows. Given trial values % and Во of the parameters required, a trial value 


о Of the logit is obtained using (5); it is 1, = 9 + Вох; , and corresponds to 
an estimate р, of Р, given by 


P» = [1 + exp (-1)]7*, 
Using this result, and retaining only the first term in à Taylor expansion, 
Do — Р, = (h — 1)Р,0, . 
Substituting this approximation in (11) and (12), 
differentials 54 and 58, which are the corrections t 
parameters, are given by the equations 
(13) Enp- Ут = Dw) + 88 (2 ша), 
E npa, — У nipt: = б D> шт) + 8B C07 wat, 


Where w; = родо , Corresponding to x, . Solving these equations the correc- 
tions are found to be 


it will be seen that the 
0 the trial values of the 


3. пра — Y aa, У wie np, — 2; Уза 
py бе У wi — o> шта Ут, 


апа 


(15) в = UP — Enp- a T 
ш; 


Anscombe [1] points out that it iterations to re- 
caleulate the coefficients on the right-hand si 
the expressions Do np; and фр 
iteration only the terms 27 тұ) and 

procedure, Anscombe (Ш, p. 461) remarks, “is to p 
Which is exactly modelled оп 


Bliss for fitting the integrated-norma] law, 
Statistics are available." 


is unnecessary in further 


dure due to Fisher and 
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Estimates of the variance of В, 4, and à are given [3] by the formulas 


var (B) = 1/52 wa; — 3); 
var (4 = (1/5; w) + ' var (8); 
var (В) = [var (6) + var (8)(@ — ®)°]/@'; 


where 2 = У) w;z,/2, wi. 
A quick graphical method for estimating « and 8 has recently been 


provided by Hodges [10]. 


(16) 


An Example 


The data used in the example are taken from Ferguson’s article [8] and 
are the same data as employed by Finney [9] in his example using probit 
analysis. Omitting the two extreme values of x, where the proportions may 


TABLE 1 


Proportions (р) Passing an Item at Different Levels of Ability (x) 


ч n Pi тірі 
-1.94 15 0.55 4.95 
-1.16 25 0.56 14.00 
-0.58 45 0.70 30.10 

0.00 50 0.94 41.00 
0.58 45 0.95 40.85 

1.16 25 0.96 24.00 

1.94 15 0.95 15.95 


fne = 114.85 Enpa; = 35.2950 


of the parameters were read from а rough 
= log, p;/4; , Where а; = 1 — P: , as 
giving the equation l214cr4- 
Iculated; these were converted 


be unreliable, estimates ао and Bo 
plot of the variate x; as abscissa and l; 
ordinate. These were 2.0 and 1.4 respectively, 
2.0. Values of lo for each x-value were now cal 
to p,-values using the table prepared by Berkson [4], and from the same table 
the weights pogo were read. These data are tabulated in Table 2, while below 
the table the quantities required for substitution in (14) and (15) are recorded. 
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TABLE 2 


Deta for the First Iteration 


5 6 Po 24, nS, = “%% 
-1.94 -0.72 0.32739 0.22021 4.91085 3.30515 
-1.16 0.38 0.59387 0.24119 14.84615 6.02975 
-0.58 1.19 0.76674 0.17885 32.96982 7.69055 

0.00 2.00 0.88080 0.10499 44.04000 5.24950 

0.58 2.81 0.94521 0.05356 40.55803 2.30308 

1.16 3.62 0.97392 0.02540 24.34800 0.63500 

1.94 4.12 0.99116 0.00876 14.86740 0.15140 


Ev, = 25.34243 En, = 17654085 


Enjjo, = 34,73832 
Znan - Enj$, = -1.69085 
Enpi, узур, = 0.55660 


=== E MENS 


Уч; = -15.535638 
Хч? = 25.25622 
(Enx)? = 241.36226 


iteration are found to be ôĝ = 

corrections have nonzero entries o 
iterations are unnecessary 
B and à now are 1.3739 and 1.9196 


using the probit method, 
ely. While he considered 
it appears that it would 
of demonstrating con- 
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vergence and in order to get a more accurate answer, had he performed at 


least one further iteration. 
Using (16), estimates of the standard errors of B, &, and f are found to be 


S.E. of Ê = 0.0540, S.E. of = 0.2014, S.E. of р = 0.1956. 


The estimate 1.3739 of 8, the gradient of the logit line, is a measure of 
the diserimination value of the item. When compared with 0.0540, the 
estimate of its standard error, it is found to be highly significant. 
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- THREE MULTIVARIATE MODELS: 
FACTOR ANALYSIS, LATENT STRUCTURE ANALYSIS, AND 
LATENT PROFILE ANALYSIS* 


W. А. Grssont 
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The factor analysis model and Lazarsfeld’s latent structure scheme 
erived to show how the latter 


for analyzing dichotomous attributes are 4 T 4 
mmunality esti- 


model avoids three knotty problems in factor analysis: co i 
mation, rotation, and curvilinearity. Then the latent structure model is 


generalized into latent profile analysis for the study of interrelations among 
quantitative measures. Four latent profile exam les are presented and dis- 
cussed in terms of their limitations and the pro Jems of latent metric and 
dimensionality thereby raised. The ossibility of treating higher order empiri- 
cal relations in a manner paralleling their various uses in the latent structure 
model is indicated. 


At an early point in М ultiple-Factor Analysis ([18), P- 70), Thurstone 


remarks: 

ss with the analytical 
mit ourselves to them 
f entirely differ- 
rement and 


It would be unfortunate if some initial succe 


methods to be described here should lead us to com 
with such force of habit as to disregard the development o! 
ent constructs that may be indicated by improvements in measu: 


by inconsistencies between theory and experiment. 


at statement to heart. | 
r analysis model will be sketched, noting 


dural problems: (i) how to estimate 


This paper is an attempt to take th: 

First the derivation of the facto 
three inherent conceptual and proce } if 
communalities in the event that only shared variance is to be analyzed, (ii) 
how to resolve rotational indeterminacy; and (iii) what to do те extra 
linear factors that are forced to emerge when nonlinearities овса in the data. 
Some recent concepts for the multidimensional analysis of qualitative data— 
the latent structure model of Lazarsfeld [15]—are considered, with special 


í 9 in factor analysis. Next these 
refere г ling of the trouble spots in fac 4 
new vanis n eem ]ternative Way of analyzing the 


zed to produce an алт, 
interrelations among quantitative measures. This 18 the tac 
АП three of these mo 


dels are discussed strictly from the езү" 
ae ; à 
sample statistics. The problem of generalizing 10 à population of whieh 521 


" by Green [12]. 
“The latter model is antie P рег was сотр ШЫ Lo for Advanced Study 
+The major portion of this paper expressed are those of the author and are not to 


кше Baha Рае The o Department of the Army policy. 


be construed as reflecting 
229 


ipated in а! 
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sample may be considered representative is not taken up for any of the three 
x. examples of the latent profile model are given. Two of them are 
fietitious but their form closely parallels the corresponding two empirical 
examples. These examples are discussed from the viewpoint of the further 
work they suggest concerning the metric and dimensionality of the latent 
space. The use of higher order empirical relations for testing fit and for 
further particularization of the latent profile model is briefly diseussed, and 
the possibility of close parallelism between developments in the latent 
structure and latent profile models is pointed out. The paper concludes with 


а plea for continued flexibility in the choice of multivariate models as new 
and improved ones appear. 


Some Problems in Factor A nalysis 


The fundamental postulate of factor analysis ([18], p. 63) is expressed in 
the simple linear equation 


(0 Zi- 


to correlated factors in апу given analy 
will not be introduced here, how 
discussion without changing the 
weights descriptive of test jand ir 

Straightforward summationa’ 


actors 
te the 


(1) are a set of 
ivariant for individuals, 


l algebra and the independence of fac 
lead directly from (1) to the basic equation of factor analysis ([18], р. 78 
(2) Tikt = Qj + айы + +++ + [70 

Thus 7), , the correlation betw 
bilinear function of the 
factor loadings, 


tors 


): 


15 expressed аз а 
"These а’з, 
ations between tests 


simple 
also known as 
апа factors, 


how no 
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deal with those elements in the correlation matrix that have repeated sub- 
seripts—the diagonal r;;'s. То take these as perfect self-correlations of unity 
would amount to trying to analyze all of the variance of every test, including 
the unreliable part. To insert the test reliabilities into the diagonal cells would 
imply an interest in analyzing all of the “true” or repeatable variance of the 
tests, including that specific to each test and unrelated to other tests in the 
battery. More commonly preferred among factor analysts is to attempt to 
analyze only that part of a test’s variance—its communality—that is shared 
with other tests in the battery. Communalities could be defined alternatively 
as those portions of the self-correlations accounted for by the factors that 
suffice to account for the between-test correlations. In any event, the com- 


munalities are not empirically given and yet are needed at the start for maxi- 


mum efficiency of solution. In principle the communalities could be determined 
lied to the empirically given 


by certain operations (cf. [18], pP- 294-307) арр 
side entries in the correlation matrix, but these operations are usually so 
time consuming that a successive approximations approach is often sub- 
stituted. Rough communality estimates are used to obtain an initial factorial 
solution, which in turn provides improved estimate 


в for a second cycle of 
the same kind, and so on until the communalities are sufficiently "stabilized." 
It fortunately happens that with 


large test batteries little or no iterating 
of this kind is needed, but for small batteries Sev 


eral cycles may be required 
before the communality problem is 


adequately resolved. Ж 4 
А second and more important problem in factor analysis is the inherent 


partial indeterminacy of the ав that is known as the rotational problem. It 
is most easily understood in terms of q-dimensional geometry. The a г Гог 
any test 7 may be thought of as the projections of a point j on a set о NE 
ordinate axes in q-space. The table of factor Joadings for the s tests tl "m 
defines a. configuration of s points in terms of their AT АГ En 
mensional reference frame. But the equations of factor je y Ex i: с, cd 


do not indicate which position of the reference fr: п 
number of possibilities in the same q-space, is to be preferred. Only the origin 


of the coordinate system is fixed, so that the reference frame can be rotated 


i i i figuration 
reely Чт iti other without distortion of the confi 
freely from any position to any S һә a's change Metab roads wee 


of points defined by it. Natural Е А 
always in such а Ei as to preserve the spatial их ee 
points they define. Many ways of resolving this qu es mol К» 
have been proposed. Probably the most Jy н >? ыы aa 
principle (18), рр. 181-193), which strives to р ot A loacings, Май 

of the tests by maximizing the number of Ке. 10 x кы d d 
of these proposals (especially those centering around зе D intuition 
principle) involve heavy computing 10 relatively ape : 


ads, 
or both; many are debated or debatable; all are in the nature of afterthoughts 
й uations 


that are not built into the ed defining the model. 
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Another perplexing problem in the factor analysis model is the paradox 
of difficulty factors [2, 4, 13, 20]. If factor analysis is applied to E battery of 
tests varying widely in difficulty but quite obviously measuring but one 
underlying trait, the result is not one but several "factors"—one for each 
level of difficulty. This is generally attributed to curvilinear relations among 
tests markedly different in difficulty, such curvilinearity being forced by the 
differential skewness of the score distributions. Note, however, that it is 
only the relations between tests and factors, as indicated by (1), that the 
factor model explicitly restricts to linear form. 

The record of empirical fruitfulness (or lack thereof) of factor analysis 
is not at issue here. Nor are misapplications of it pertinent to this discussion. 
The other two models to be discussed here are meant to be put to much the 
same use, and they may very well suffer the same kind of misuse. 


Some New Concepts: Latent Structure Analysis 


Only one variety of latent str 
crete class model—is discussed here. 

Latent structure analysis [15] is Lazarsfeld’ 
the interrelations among dichotomous attributes, 


on a survey questionnaire. It is based on linear r 
following kind: 


ucture analysis—that known as the dis- 


8 technique for analyzing 
such as the item responses 
ecruitment equations of the 


w= m Ещ eR, 


(8) Ni = три; + рә; + +++ + тре, 
Nik = р + Тара ++ + Tapas , 


"ы = pa, + пры + ++ 


+ паа » ete. 
The quantities on the left are empirically given or 
indicate the number of people in the entire sample, n, 
а single item, n; , the number endorsi 

The quantities on 


80 on. 'The latent probability p; i 
class 1 who endorse item j, 
endorse both items j and К, 
manifest joint endorsements 
be noted, in passing, that i being a set of 
are intrinsically linear, whi ini 


only because it was made so. 


—— "-— 
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The manifest-latent distinetion that is so prominent in latent structure 
théory is of course central in factor analysis as well. There the tests and 
their intercorrelations are manifest, and the factors and the loadings thereon 
are latent. The same distinetion also appears in the latent profile model to 
be discussed later. 


The preceding recruitment equations are equally valid for any method 


of classification and for any number of latent classes. The next step is to 
invoke a pertinent basis for classification. This is the core of the latent struc- 
ture model. It is quite relevant to require that each latent class exhibit 
homogeneity with respect to any underlying dimensions that may be re- 
sponsible for the manifest interrelations. Perfect homogeneity is not crucial, 
so long as deviations from the class norm are random. Such random deviations 
are of course uncorrelated within the class. Thus it is sufficient to require 
that each latent class be homogeneous enough, with respect to any and all 


such latent dimensions, so that all item responses within the class are independ- 


ent in the coin-tossing sense. This intraclass independence is expressed in the 
following equations. 


* ‚рак = PaiPak › 


(4) Piik = PiiPik , Paik = Рг › `` 
+ ‚Рем = РаРафа › ete. 


Тым = mp › рны = P2iP2xPat › ave 


The substitution of (4) into (3) yields the basic equations of latent 


structure analysis ({15], p. 385). 


mp tng tots Fe» 

4o + Фа › 

+з, + тараа › 

F 55 + пурораРа » etc. 


І 


т 
(5) т, = три; + р 
тк = тар, + npo 


ты = приори + тора Ре 
i joi uencies are accounted for in terms of 
Thus all of the manifest joint frequ со teas 


1% ра ameters, 4 i п! q late 

q 84 atent у ter: class sizes 8: d lat e 

4 s ; : ) for auti of the s items. The successive levels of manifest fre- 
ый s Pai 


quencies (n, 4; , ft. 640) number, respectively, E P Lam ТӨ иң , T 
coefficients in the binomial expansion (c + iim ES ahis model. | 
number of equations relating manifest oe 


е ta e, а acto ysis, i / 1 tions for 
Th k her i - analysis, is to solve the basic equa. 
the u im n аын 1 latent structure solutions already 
he unknow 


i 7] avoid the use of any joint frequencies 
exist. The most recent of these [1, 7] e dd doses qeaemes 


1 1 ji s Pii 224 ` 
= So күсе э i Коко to the communalities of factor analysis, 
"tii pee manifest To interpret them as equivalent to the corresponding 
in no г 


ters. Severa 
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lower order joint frequencies without repeated subscripts (1.е., n;; = m; , 
Tiit = Nix у Т; = n; , etc.) would be analogous to the use of unit self-corre- 
lations in factor analysis. Instead, the latent structure model usually treats 
these elements with repeated subscripts as stemming from 
leading to, the latent parameters of the model 
the manifest data without repeated subscripts. 
The Anderson latent structure soluti 
eliminate the latent structure analo; 
the use of manifest data w. 
cursory investigation indic 
factor model to make use 


; rather than 
that suffices to account for 


on and an earlier one by Green [11] 
gue of the rotational problem through 
ith more than two subscripts, such as туы. (А 
ates that it might be self-contradictory for the 


among factor scores woul 
destroy the basic linear p 


These higher order data select. 


ower order manifest frequencies equally well, the 
t. Another early latent structure solution [5, 6, or 8] 
Severe) reduction of rotational indeterminacy without 
the expense of obtaining higher order data. This is accomplished by capital- 


at the latent parameters, being probabilities, can 


resent. Note that the derivation. 
of the latent structure model embodi 
either among t 


д 1 г between the 
Қа ени them and the latent 


the size, sums of 
) Products for t i Г 
sponding subgroup Bean or the entire sample аг 


2 А е related to corre- 
mg simple way. 


* the various subgroups. 
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N=M cnc $M; 


P Es а-а ee 


п nt 


E х.Х. = У Х.Х. + У! ХнЖь- oe + Ух » 


(6) 


тї 


У Xa4XaXa- У XagXaXa 
+ У хьхаха + + У ХаХаХа, ete. 


All summations in (6) are over individuals. The summations on the left 
are over the entire sample, while those on the right are over the members of 
X; is the score of individual 2 on test j, and it may be 


in raw, deviational, standard, or any other derived units. The same is true 


of Xir, Ха, ete. 

It is convenient to designate the various summations in (6) by the 
letter m with appropriate subseripts. Thus т; and т, represent, respectively, 
the sum of scores on test j and the sum of products of scores on tests j and k, 
each sum being for the entire sample. m; and mu;, , On the other hand, 
stand for the same things in subgroup 1 only, and so on. Equations (6) then 


become 
mm + т. ter №, 
(7) т; = т; + Ma; fees Е та, 
mg = manc Тар Tee F Mairo 
туы = Majer + Тат Feee F Marr, Cte 


The-m’s in (7) will be referred to as product moments of various orders. 
The product moments in the third line, involving two tests, will be said to 
be of second order, while those for three tests are of third order, and so on. 
For consistency, the m’s involving only one test j will be called пас 
product moments, and п, the sample size, could be called газа ег 
produet moment for the sample. The m's on the left in (7) will be called 


sample product moments, while those on the right are subgroup product 

moments. j 1 ы j $ 
It should be stressed that (6) and (7) in no way restrict Te E 

pirical data or the method of classification into subgroups. It w : e th e 

burden of the next two sections to establish & иеа сар that wi 

convert these recruitment equations into a mathematical model. 

em of Latent Profile Analysis 

t distribution in which the score 

lotted against his score on test j. 


The Fundamental Theor 
Consider now a two-dimensional join 
kis p 


of every person in the sample on test 
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Such a scatterplot is pictured in Figure 1. The unit of measurement for both 
tests is entirely arbitrary here. The ellipse in Figure 1 indieates only one of 
the possible shapes that the total configuration of points could have. The 
circle represents a subgroup 2, of size n, , within which the correlation between 


X 


k хы 


FIGURE 1 
А Hypothetical Scatter Diagram 


tests j and k is zero. The two lines labeled X 
the means of subgroup # on tests j and k. 
centroid of the points com 
individual 2, a member of subgroup 1 


EE, mu 
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The ј and k scores of individual i may be expressed in terms of his sub- 
group means and his deviations from those means, as follows: 
(8) Xi = X; + dj, 
(9) Xir = Xn +da. 


Then m,; , the sum of j scores for subgroup 2, is given by 


(10) ma = > Xi: 
= > (Ха + dis) 
T oe + Уа, 
= nX. 


The first term in the third line simplifies because X,; is the same for every 
member of the subgroup. The last term in the third line vanishes because itis 
а sum of deviation scores. Thus the j scores for any subgroup % contribute 
to the sum of all 7 scores as if the members of the subgroup were concentrated 
at their mean for test j. This result (by no means а new one) will next be 
generalized, in an appropriate way, to the second-order product moment, 


Miir , for subgroup 2. 
By definition and with the help of (8) and (9), mu; becomes 


(11) Min = У Хи Ха 
= 3 Gn + da + do) 
= Уха + Xo da + Xa da + dudo) 
- Ў хаха + хи X das Xa Ха 5: dg dis 


= ХХ. 


The first term in the fourth line of (11) simplifies because X,; and X,, are 


у "oup. The second and third terms in that 
the same for all members of the subgroup ОШ шылкы кше 


i 1 cause they contain sums of i: Me 
[ria ү” эү шй of the formula for the ccr ыт 
subgroup, between tests j and К. The subgroup га ыы ое > 
that correlation equal to zero. Thus the second- | ы vedere 
та , is the same as it would be if all members of иго sera өрүс 
at their centroid. This holds for any subgroup within whic j 


4 j Н . The quantit; 
TE d distinction in terminology needs to be made here. q y 
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түк is a sum of products of horizontal and vertical distances from the origin 
in Mieure 1. It will therefore be called a product moment about the origin. 
Another kind of product moment in (11) is the last term in line 4, the sum 


а set of points from their 


xceptions are the last term 
ast three terms of line 4 in (11), 
ir point of reference, 

eralized to higher order product 
on subgroup /. Not only must 
f tests j, k, and 1 uncorrelated 
e tests, a vanishing third-order 
hese restrictions its third-order 
‚ becomes 


Tug, = "ХХ Хи . 


order analogue of 
order equivalent is 


been placed at any 


al theorem of latent 
profile analysis, 


The g-order product moment, 
zero product moments of ord. 
equal to the g-order product 
placed at that centroid. 


about any point 0, of n, 
ег g and less about the. 
moment, about the point 


points having 
ir centroid, is 
O, of n, points 


The Basic Equations of Latent Profile Anal, 

The foregoing theorem provides a basis for g 

equations introduced previ 
analysis will be obvio; 


ysis 


сештепсе, and not just 


case in latent structure analysis, where withi 
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as pertaining not only to all pairs of items but also to all triplets, all quad- 
ruplets, and so on. (It is easily shown by example that higher order independ- 
ence among dichotomous attributes is not a mere logical consequence of 
uncorrelatedness between all pairs of such attributes.) 

The concept of uncorrelatedness among quantitative measures, on the 
other hand, has more often been restricted, at least among psychometricians, 
to pairs of such measures. (A statistically oriented prepublication reviewer 
has pointed out that the two kinds of independence being discussed here are 
known to statisticians as pair-wise and mutual independence.) This is not an 
intrinsic or logical difference between qualitative and quantitative statistics. 
It is rather only a historical accident that the question of higher order in- 
dependence among quantitative measures has less often arisen in psycho- 
metrics. That question arises here, for it turns out that the proper definition 


of such independence is crucial for this model. 
In the previous section the within-class uncorrelatedness between pairs 


of tests was shown to be synonymous with vanishing second-order product 
moments about the centroid of the class. This is because such product 
moments are the numerators of the formulas for the corresponding correla- 
tions. Purely by analogy, higher order within-class independence may be 
equated with the vanishing of higher order product moments about the 
centroid of the class. (The failure of such product moments to vanish would 
allow, for example, a positive correlation between tests j and Ё among class 
members with high scores on test 1, accompanied by a compensating negative 
correlation between the same two tests among class members having low 
scores on test 1. This could happen in spite of zero correlations between all 
pairs of the three tests within the class as a whole. If correlational patterns 
can differ within subdivisions of a class, then the class is not homogeneous 
even from a commonsense point of view.) Therefore let the within-class 
independence of the present model be defined as applying to all orders of 
interrelations, and as expressing itself in the vanishing of product moments 
of all orders about the centroid of the class. Then the fundamental theorem 
applies with full force to the product moments of each class, 80 that the results 
of equations (10), (11), and (12) can be used to transform (7) into the basic 


equations of latent profile analysis: 
nm + ть + tm, 


my Xa; + "Ха ctc + ты , 
+ + ХХ, 
qo cRnmXgXeXa, etc. 


т; = 

р-з z 
mj = MXX + таХ2 Хз 
niXiXuXui + тХЫХАХа 


moments (including n) are accounted 
rs—the q latent class sizes and the 


(13) 


Tk = 


"Thus for s tests the 2' manifest product 
for in terms of (q + sq) latent paramete 
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4 class averages for each of the s tests. Each latent class is therefore character- 
ized by its size and its profile of s test averages, 


j i; are the between-groups 
variances.) Nor do the latent profile equations restrict the occurrence of 


sts and underlying dimensions. 


Thus the dilemma of difficulty factors is avoided in this model. 


Two Special Cases of Latent Profile Analysis 


In the development of the latent profile equations the 
entirely arbitrary. There are, however, two kinds of test scores that deserve 
special attention. Consider first the case wher i 


in latent structure analysis 
of one and zero, 


erages of latent profile 
Tucture analysis, The 
dels. Thus the latent 
f latent profile analysis 


dividing the latent profile equations throu 5 mber of people in 
the sample. The latent profile equations then assume their 
1=р +p +: фр, 
(14) 0 = pZ; + Poo; Ros а Таби, 
Tik = DiZiZyu T 2,2,2, Tee + 2.2.2, 


"а = рб, ый, + 2.2.2000 + .. 


ak y 


diu Pal ail ц * ete. 


(in the second line) vanishes bec 
Standard scores on that test, 


standard scores, are the same 


Correlations for whic 
to account. The r 


іш аге, analogously, average $ 
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The latent profile equations in standard form have the advantage of dealing 
with magnitudes that are independent of sample size and of arbitrary score 
units. It is in standard form that the equations will be applied to the examples 


in the next four sections. 
Latent Profile Example I: A Fictitious Two-Class Case 


The fictitious manifest data in Tables 1 and 2 will serve as a first latent 
profile example. Imagine the data as resulting from the administration of 
four alternate forms of the same test (such as arithmetic) to a sample of, let 
us say, а hundred people. Suppose, further, that all six intercorrelations turn 
out to be .50, so that every two-dimensional scatter diagram, with scores 
plotted in standard units, has the appearance of an ellipse twice as long as 
wide, centered on the origin, and tilted at an angle of 45 degrees. There are 
four three-dimensional scatter diagrams. Each is approximately egg-shaped, 
and, being symmetric about the origin, yields a third-order manifest product 


moment of zero. 

Tables 1 and 2 display 
The upper left entry in Table 1 
latent profile equations in stan 


the necessary manifest data in a convenient way. 
is the first term in the first line of (14), the 
dard form. The other entries in row and 


column 0 of Table 1 are the means of standard scores for each of the four 
tests—the left-hand term in the second line of (14). The remaining cells of 
Table 1 contain the test intercorrelations—the manifest data in the third 
line of (14). 

Table 2 summarizes the manifest data of first, second, and third order. 
The upper left entry is the sum of the four means of standard scores. Each 
of the other entries in row or column 0 is the sum of the four correlations 
(including 7;;) involving the associated test. Every other cell in Table 2 
contains the sum of the four third-order manifest product moments (including 
rjj, and ту) for the corresponding pair of tests. 

For convenience of exposition in both fictitious examples in this paper, 
all elements with repeated subscripts (such as rj; , Ге, and гу) are treated 
as known. Their values are, in fact, easily inferred from the simple form of 
the manifest data, but this would not be true generally, even for all sets of 
fictitious data. In this first example, all r;; are .50, all ти» are zero, and all 
Тын ате zero. В A 

Tables 1 and 2 have been labeled Ё and R, respectively. This is con- 
venient notation for any such display of manifest data, and it will be used 

а distinction must be made 


in all examples. In any latent profile solution, be ma 
between the given and the fitted R and R, , the latter pur of tables indicating 


what the former should be in order to be completely accounted for by the 
solution. In both fictitious exam the given and the fitted 


ples in this paper, th 
manifest data, the latter computed from the solution by means of (14), 
are identical and hence need not be compared. In the two empirical examples, 
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however, the comparison will be made when possible in order to appraise the 
adequacy of the solution. 

The latent profile solution for the present example, obtained by applying 
the algebra of the latent structure solution of Green [11], is shown in Table 3. 
Each of the two latent classes is defined in terms of its relative size and its 
latent profile—the complete set of average test scores for its members. 
Apparently Class I consists of those who are poor at arithmetie, while Class 
II contains the good arithmetic students. 

A fruitful way to visualize this latent profile solution is in terms of the 
regressions of the tests on the latent continuum of arithmetic ability. Such 
a graph of mean test score, Y, against position along the latent continuum, 
X, is shown in Figure 2. Here the regressions of all four tests on the latent 


Y 
1.00 
Tests 
1-4 
Class I 
| ЖА 
—|.00 1 
Class II 


Tests 
l-4 


-1.00 
FIGURE 2 


Tests on Latent Continuum for A Fictitious Two-Class Case 


Regression of 


Y and X are expressed in standard 
line is also the correlation between 
“factor,” and for the linear 
ns turn out to be exactly 


continuum are identical. In Figure 2 both 


units, so that the slope of the regression 
Y and X. This is à correlation between test and 


. correlatio 
regressions of the present example, these 
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the same (4/2/2) as the factor loadings that would result from a factor 
analysis of the correlations in Table 1. This simple correspondence between 
the two models will vanish, however, as soon as any of the regressions become 
nonlinear. 


Latent Profile Example YI: An Empirical Two-Class Case 


Ав a second latent profile example, consider the given В in Table 4. 
That table is а simple modification of a table of intercorrelations among 
nine reading tests previously reported by Davis [3]. The only modification 
was to border Davis’ table with the 0 column and row. The intercorrelations 
were based on 421 cases. 

A letter from Davis has indicated that the талу scores that would be 
needed for the computation of R, are no longer available. In the absence of 
higher order data which would provide a unique solution, it was necessary 
here to adapt some factoring and rotational procedures that were involved 
in an early approximate latent structure solution [5, 6, or 8] in order to obtain 
ution. For this purpose a factorization of 
the Davis data by Thurstone [19] was used. Thurstone’s analysis indicated 
that one factor was sufficient to account for the data, all but three of the 
discrepancies between ‘given and fitted correlations being less than 04, and 
the largest being .07. It is with exactly these same discrepancies that the 
present latent profile solution accounts for the intercorrelations. 

An approximate latent profile solution for the Davis data is Shown in 


Table 5. This solution is obtainable by resolving the rotational problem 
with any one of the following three assumptions: 


() that the two latent classes are equ 

(ii) that the two latent profi 
sign; or 

(iii) that the given R, , 
having nonzero entries only 


al in size; 
les are identical except for reversed algebraic 


if available, would be like tha. 


t of Example I in 
in its 0 column and TOW. 


The solution in Table 5 generat 


es, by means of (14), a fitted By 
form indicated in assumption 3. 


having the 


be shown to 
that the slopes of the re- 
remain constant regardless of 
This is but one illustration of 
nt profile models are mutually 
of both models are not violated. 


when both axes are in standard units, 
how the rotational indeterminacy is resolved. 


late 
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FIGURE 3 
Regressions of Tests on Latent Continuum for Nine Reading Tests 
Latent Profile Example III: A Fictitious Three-Class Case 
in Tables 6 and 7 will provide a first illustration 
s the problem of difficulty factors. Imagine 
abulary tests, tests 4 and 5 as two hard 
bulary test of intermediate difficulty. 


n a hundred cases. 
s of this fictitious data, it 


The fictitious R and Rı 
of how latent profile analysis handle: 
tests 1 and 2 as being two easy VOC 
vocabulary tests, and 3 аз 8 voca 


Again assume the data are based upo с 
Before proceeding to the latent profile analysi і 
will be instructive to examine the results of afactor analysis of the correlations 


in Table 6. The simple structure factor analytic solution with correlated 
factors is given in Table 8. The entries in that table are the correlations 
between the five tests and the two factors, A and B. The correlation between 
the two factors, тав › 18 :38- If the usual rules for interpreting factors were 
followed unquestioningly here, the conclusion would be that the two factors 
are knowledge of easy words, Jedge of hard words, B, and that 


А, and know о 
the two abilities are relatively independent. This 18 absurd. d 
The unique, perfectly fitt file solution for this example, 


ing latent рго ; : 
again obtained from В and R, by the same algebra as 15 used in the latent 
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ТАВТЕ 6 TABIE 8 


R for & Fictitious Three-Class Case Simple Structure Factor Analysis 


Solution for Correlations in Table 6 
See en ыкка эх 
== = ————ЄЄү;—ү———— 


= 
t Numbi Test Factors 
Be о 1 Тез 2 = 5 2 №. А B 
Е 5 “B 
0 1.00 .00 .00 .00 К] .00 1 .82 .00 
1 .00 «T5 415 «50 .25 .25 2 „82 .00 
2 .00 “75 ШЕ] .20 .25 +25, 5 № hl 
э .00 50 450 420 .50 -50 4 +00 .8e 
b 0.00 .25 25 „5% A5 .5 > 200 82 
5.9 5 45 Бо т +75 Тір = 33 
SSS — ———_ 
TABLE 7 TABIE 9 


В; for a Fictitious Three-Class Case latent Profile Solution for 


8 Fictitious Three.Class Case 
C a I MU 
Test Test Number 
No. SS ee a DEL] 

о “0 2.50 2.50 2.50 2.50 2.50 
1 2.50 -2.50 -2.50 -1.25 400 400 
2 2.50 -2.50 -2.50 -1.25 +00 +00 
5 2.50 -1.25 -1.25 “0 1.25 1.5 
% 
2 


Test 
No. 


latent Class 
i 


її ill 


2.50 400 40 1.25 2.50 2.50 


250 .9 .0 1.25 2.0 2.50 Class Sizes — .25 50 25 
m PME pel 


Structure solution of Green [11], is shown in Tabl 
standard units, the regressions implied by Table 9 a 


equal spacing of the latent classes along the sin 
vocabulary knowledge. 


The contour of the various ге 


e 9. Figure 4 Shows, in 
nd by the assumption of 
gle latent continuum of 


gressions in Fi 


Latent Profile Example ТҮ: An Empirical Three-Class Case 


example. Aside 
rsion of a table 
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FIGURE 4 

Class Case 


Regressions of Tests on Latent Continuum for A Fictitious Three- 


tion for the Ferguson example 
was obtained by а rotation of the factori- 
28). Here again it would be absurd to 


that the data must be thought of in 
d low-level verbal 


The simple structure correlated factor solu 


is shown in Table 12. That solution 
zation reported by Ferguson (4], р. 3 


conclude, according to the usual rules, e 
terms of two relatively independent factors—high-level an 


intelligence. 
A letter from Ferguson has indicated that the raw scores are no longer 
available for the computation of higher order manifest product moments. 
It therefore was necessary to approximate the latent profile solution by 
means of procedures similar to those used in the Davis example. For this 
purpose Ferguson's factorization of his correlations was used. His two factors 
accounted for the correlations with discrepancies not exceeding .03, and the 
present latent profile solution fits the correlations 1n eactly the same way. 
The rotational indeterminacy here was resolved in two stages. ' The 
first step was to rotate Ferguson’s factorization, with attention being given 
only to subtests 1 and 6, into maximum correspondence with the initial 
factorization for the first an in Example ш. 


d last tests The latter factori- 
zation was a part of the late lution for that example. In Example 
III the unique solution was © 


nt profile 80 DE NS 
btained by applying, to the initial factorization 
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TABLE 10 


Given Correlations for Six 
Verbal Intelligence Sub-Tests 


а mteligence вый 00 - 


TABLE 11 
Fitted R for Six Verbal Intelligence Sub-Tests 


Test Test Number 
No. о 


Е 1 2 3 ш 2 5 
O .0 4,32 


Мә 2.80 18. һат зт 
la bbe -3.95 -2.98 -3.07 -1.7} . oh 
4.9 -3.93 -3.37 -2.30 -2.39 -1.02 15 


3 

2 

5 180 -2.98 -2.30 - .99 -1.08 45 1.75 
+ 158 -3.07 -2.39 -1.08 -1.16 426 1.70 
5 MT -1.74 -1.02 45 426 1.52 2.71 
6 


3.77 - 2% 45 1.75 1.70 2.11 5.57 


EE Me SQ 
of R, a rotation completely specified b 


Second rotational Stage in the present 
the Example III solu 


quences of this rotational solution are 


(i) The relative class Sizes are the 


Gi) The regressions of all six su 
ascending. 


(ii) Assuming equal spacing of the classes 


the regressions of subtests 1 and 6 hay 
equal but opposite in direction, 


(iv) The form of the fitted R, is similar 
В, of Example III. 


е eur 


the following, 


to that of the given 


TABLE 12 


Simple Structure Factor Analysis 
Solution for Correlations in Table 10 


z a = 
No. А B 

1 .86 +00 

2 15 15 

3 -55 +43 

4 .57 42 

5 +30 +64 

6 +00 81 

TABIE 15 


Approximate Latent Profile Solution 
for Six Verbal 


Intelligence Sub-Tests 


latent Class 
Ee Ic YE —HWri- 


1 -1.64 151 62 

-1.56  .56 8, 

Class -1.38 .от 1,24 
Меапв 


y the higher order data in В, . The 


approximate solution was to imi 
tion by using exactly the 


tate 


Same rotation, The conse- 


same as in Example ITI. 
btests on the late 


nt continuum are 


along the latent continuum, 


vatures that are approximately 


and fitted 
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FIGURE 5 
Regressions of Subtests on Latent 
Continuum for Six Verbal Intelligence Subtests 
dy the nonuniqueness of this latent profile solution. 
lied to all of the alternative solutions that were 
scending for all subtests. 
nd in class averages could 
the ordering of curvatures 


computed in order to stu 
Only one restriction app 
tried, namely, that the resulting regressions be а. 
Within this restriction large changes in class sizes а 


be brought about, but never in such a way as to alter 
among the regressions. It should be added that only with very strained 


assumptions about the spacing of the three classes along the latent continuum 
could the regressions of both subtests 1 and 6 be made to curve in the same 
direction. 

Table 11 shows the 
Table 13. A comparison of 
the fitted higher order ma 
examples. 


the approximate solution in 
reveal the similarities between 
two three-class latent profile 


fitted Rı implied by 
Tables 7 and 11 will 
nifest data for the 


Discussion 


ons to the foregoing late 
ly. These are in additio 


There are two limitati nt profile examples that 
n to the indeterminacy 


should be discussed explicit! 
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i г ifest data in the case of the two 
the absence of higher order manifes i 
[xe fore The first is the lack of a scale of measurement for the 


moments of still higher order, a 


nogeneous in content but graded in difficulty) 
ould require two 


many аз (q — 1) 
interpretation, Subsequent work [ef. 10] 


sional examples and with the problems of 
dimensionality and metrie that arise in their interpretation. 


The reader may have noticed that in all four latent profile examples 
no mention was made of а need for manifest data of order higher than the 
third. Even for th 


e two empirical examples a unique solution would not 
have required the use of such higher order manifest data, These higher 
order data therefor testing the assumption of higher 
order within- be done by comparing the given 
fitted values as generated from 

of equations (18) or (14). Alter- 
natively it could be argued that, j i т requires data above 


ostulate within-class independence beyond 


| 
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that order. From this parsimonious viewpoint, the latent profile equations 
need extend no higher than third order, and it would be inappropriate to 
think of using higher order data to test the fit of the model. However, it 
might then be more important to test the adequacy of the solution by relating 
it to variables not included in the original analysis. (Ап empirical example in 
[8] gives an illustration of how this could be carried out in latent structure 
analysis, where the very same argument over the use or non-use of higher 
order equations not needed in а particular solution can be made.) 

The latent profile equations that have been derived and illustrated here 
are analogous to only one form of latent structure analysis—that known 
as the discrete class case. The analysis divides the sample into a small number 
of discrete classes possessing second- and third-order within-class independ- 
ence, and stops there. Other varieties of latent structure analysis, one of 
which has already been referred to, have gone further in stipulating the 
algebraie form of the regressions (the so-called trace lines [cf. 15] of latent 
structure analysis) or of the set of class sizes, or of both. Always, however, 
the postulate of within-class independence js retained. Usually these further 
restraints require within-class independence of higher than third order, so 
that the corresponding higher orders of manifest data become directly 
involved in the solution. Most of these variants of latent structure analysis 
are readily translated into latent profile terms. In fact, the analogy between 
the two models is so close that almost whatever progress is made in the 
various solutions for one model is convertible into а corresponding advance 


for the other. 
Conclusion 


the derivation of the factor analysis and latent structure 
the latter can be generalized for analyzing 
in a way that avoids some of 


After outlining 
models, this paper has shown how 
the interrelations among quantitative measures 
the troublesome problems of factor analysis. The resulting latent profile 
model is applied to some simple fictitious and empirical data to illustrate its 
use. Because such applications may seem to show some promise, it 1s perhaps 
appropriate and not premature to conclude this paper merely by broadening 
the reference for the admonition ([18], р. 70) with which it began. 

i initial success with the analytical 

2 des ты И to commit ourselves to them with 

geld d the development of entirely different con- 


such force of habit as to disregard th 1 y 
panem that may be indicated by improvements in measurement and by 


inconsistencies between theory and experiment. 
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STRATEGIES AND LEARNING MODELS 
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A class of strategies is defined, each member of which possesses a certain 
lausibility. If a subject follows any strategy in this class in a two-choice 
earning experiment of the type dealt with by the Estes model, the subject's 

long-run behavior will be the same as that predicted by the Estes model. 


Consider a partial reinforcement experiment in which there are two 


alternatives of behavior А, and А» . If А, is chosen on a particular trial, 
reward occurs with probability ті; if А» is chosen, reward occurs with prob- 
the subject chooses А, on trial /. 


ability т. Let р, (0 be the probability | 

In special cases of the Estes stimulus-sampling model Estes ([1], ch. 9, 
р. 134) has found that if p, (t) is the expected mean probability of choosing 
А, then 
@ B(n-4-1 = 61 т) 0 — 20 + Om + 6r2)pi(n), (0 а constant) 


and, letting p, = limgi(/) then, 
1-ш 


1- то 
à ДАРЕ 
“do better" (in terms of maximizing his expec- 
eward probability, 


tati iq uicem т the alternative having greater? 

ation oosing the alte. у 

т. АРБАУ of this type of behavior (Flood [1], ch. 18). 
Simon [2] has shown that the Estes result can be derived ш йш шз 

tion that the subject is behaving rationally in a certain game-theoretie sense 

and is attempting to minimize his 
In this note, 


“regret.” 


lim p(t) = 2—ту— т 
зе 


ng to one of a certain 


the subject is adheri 
om described by having 


per of this class may be 
policy, i.e.: 
у reward, the same choice will be made 


is derived from the assumption 
class of strategies |5,). А mem 
the subject decide in advance on 4 


i ice is followed b 4 
(i) when а choice 18 (oath probability 1); 


оп the next trial all the time 
253 
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Gi) when a choice is not followed by reward, the same choice will be 
made on the next trial with some fixed probability y = 1. 


The above axioms may be interpreted in several ways, e.g., the subject, 
equips himself with а mechanism such as а barrel of balls, y of which are 
marked “repeat,” and he samples the barrel with replacement each time he 
is not rewarded, to decide what to do on the next trial. Or the subject has 
some built-in mechanism which affects his behav; 


ior in the same way. At any 
rate, each real number y, 0 < y < 1, determines a strategy, call it 5. . 
Тһе equation 


(3) pln) = mpi(n — )+(1— mpn — Dy 


+AU- т) — р — Па — 7) 
may be regarded аз a formal statement of axioms (i) and 
is following strategy S, he will choose A, on trial n: 


(a) if he chose A, on the trial п — 1 and was rewarded, (which occurs 
with probability mp, (n — 1); 


Ц 
(b) if he chose А, on the trial п — l and was not rewarded, and the 
mechanism tells him to choose А, on the next trial, (which occurs with 
probability (1 — m)p,(m — 1)y); 
(с) if he chose A, on the trial 7 
ism tells him to choose A, on the 


й—лзл)П1—р(%—1](1— 7)) 


(ii), for if the subject, 


v — 1, was not rewarded, and the mechan- 
next trial, (which occurs with probability 


The above difference equation is of the 


same type as the one obtained 
from the stimulus-sampling model and 
(4) йар) = т | 
tao = т — т, 


This time, the limit is independent of y and so each member S 

{S,} yields the same limiting behavior in 

аз well as that of Bush, Mostell 
Except for the bars over р\(п) and р, (п — 1), 

and only if 9 = 1 — ү. From this, one may conclude 


one of the strategies S, defined by (1) and (2), 
will be the same as the m. i i 


+ Of the class 
agreement with the Estes model 
er, and Thompson ([1], ch. 8). 


ean behavior predicted by the stimulus-sampling 
model if and only if y, the probability of repeating after nonreward, is equal 
to 1 — 6; to stretch a point, the constant value at; 


tributed to 6 may be para- 


phrased “hope springs eternal, with probability 1 — 9,” 


A Generalization 
Each member of the class of strate; 


е ] gies just indicated determines the 
subject’s choice on the basis of what hap) 


pened on the Preceding trial. There 
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are also strategies in which the subject takes into account what has happened 
on the preceding k trials, which also lead to the same asymptote. 

To construct such strategies, let k be a fixed positive integer. Also let 
Wy, c, W, Бе k nonnegative real numbers such that Ха; = 1. Let 
qi, 77 , Ya Бе k real numbers such that 0 € y; < 1. Let 


8; = mpa — i) + (1 — mpa ду: + (1 — тӘП — pin — 11 — т). 
Then let 
pln) = » ш; 6: . 


This not only defines р, (п), the probability of choosing А, on the trial n, 
but yields a strategy, i.e., the subject looks at the last X trials, each of which 
has a "weight." Thus, if the subject chose А, on the trial n — i and was 
rewarded, that trial contributes weight tw; - If the subject chose A, and was 
not rewarded on the trial n — 2, then that trial contributes weight шл. If 
the subject chose A, and was not rewarded on the trial n m that trial con- 
tributes weight w,(1 — y:). Then the probability of choosing A, on the trial 
n, given a particular sequence of k preceding outcomes, 18 the sum of the 
weights contributed. For example, if yı = Y2 = сс = Y = 0, iben each 
trial contributes weight w; if the subject chose ED and was rewarded, or 
chose A, and was not rewarded; otherwise, the trial contributes 0 weight to 
the probability of choosing 4; on the trial n. 
Now, it is easy to show that 
DSt, 
lim piln) 3 2 = тр — т 
Proof. lim,.« рз (п) exists. Let lim,.. рап) = pi ; then pr is the solution 
of the equation } 
J S д i > wil т). 
Pi = түрі > w+ - ту)р! = Wits ie 


Let 

k 

y= 25 WY: ; 

4-1 

then 
p = т} +a- та)руу ЕЕ т) — Ра — ” 
апа у + 1, and hence 
„Шла =, 


Duc—9-g,-— Та 
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This shows that, if the subject follows a strategy S, in the class {S,}, 
his asymptote will be the same regardless of how good his “memory” is. In 
short, he will do no better in the long run by considering, say; the last 100 


trials each time, than if he only considers the last trial ; and furthermore, his 


long-run behavior is unaffected by the choice of weights which he gives to 
each trial. 
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À NOTE ON TRYON'S MEASURE OF RELIABILITY 


ALLEN L. EDWARDS 


THE UNIVERSITY OF WASHINGTON 


. Two alternative formulas, based upon the analysis of variance, are 
given for Tryon's general form for the reliability coefficient. 


E general form for the reliability coefficient т, of an unstratified com- 
posite X, , consisting of the sum of n observations obtained for each of № 
individuals, has been published by Tryon [2]. For example, for an N X n 
matrix of observations, as shown in Table 1, with № individuals or rows and 
т columns of observations, the general form is given by 

2; 
né 
(1) Ta = KA D 
Where &,; is the average of the covariances of the entries in the n columns, 


and У, is the variance of the № row su 
Tryon shows that r,, may be calculated from any one of a number of 


algebraically identical formulas, but does not show the relationship between 
(1) and two useful alternatives based upon the analysis of variance. For 
example, (1) can also be written, in a form given by Hoyt [1], as 


тїз. 


@ ти =1- М5, , 


Where MS,, is the row X column interaction mean square and MS, is the 


mean square between rows, ог а8 


n ма.) 
G) мегі (1 - 085), 


where М8,,, is the mean square within columns. 
For the data of Table 1 Tryon reports 7, based upon 51 = 1 
In Table 2 appears the analysis of variance for the same example. Substituting 
in (2) with the appropriate values from Table 2 yields 
3.728 _ 880 
pu = 0 = 31.122 .880. 
the sum of squares within columns is 280.1 + 


For the same example, 
257 
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TABLE 1* 


Illustrative Score Matrix 


Test Samples 


Subjects 

ху Xp X3 X, Xs ae 

2 8 6 5 2 4 25 

3 10 12 v 7 1 43 

4 5 11 п 9 8 Ay 

5 6 3 o o à 10 

6 п 7 9 6 1 34 

T T г 2 ° 5 26 

8 4 1 4 4 1 20 

9 6 3 3 2 4 18 
10 6 5 1 3 ai 16 

2: 69 63 43 38 32 
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TABLE 2 


Analysis of Variance of the Scores Given in Table 1 


Source of variation 


Sum of squares df Mean square 
Between rows (subjects) 280.1 9 31.122 
Between columns (scores) 10h.2 4 26.050 
Row x column interaction 134.2 36 3.728 


Total 518.5 h 


— uu — Ы ор 


тщ 
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134.2 = 414.3, with n(N — 1) degrees of freedom. MS... is, therefore, equal 


to 414.3/45 = 9.207. Substituting with this value in (3), 


ву е 
hut ;ü at) (80; 


The two analysis of variance formulas will be shown to be algebraically 
equivalent to (1). Let the sum of squares within columns be Y, the sum 
of squares between rows be 2; s; , and the row Х column sum of squares 
be $5 a,2 , with degrees of freedom of n(N — 1), (№ — 1), and (n — 1) 
(N — 1), respectively. Division of each sum of squares by its degrees of 
freedom gives the corresponding mean squares Ms... , MS, , and М8... 
Тһе mean of the variances within columns is 


(4) Ӯ, = @а+.- + s)/n = тез = М8... 


d, in his notation, is 


Tryon’s V, is the variance of the row sums an 


(5) У, = nV, + n(n — Dé - 


The variance of the row means will then be 


(6) 52 = L p Ys + nv — 061. 


Also, ns? = MS, and, therefore, 


(7) nisi = п MS, = Ve 


bstituting with this value in (1), gives Tryon's 


Solving for б; in (5) and sul 
variance form or 


e а 
То derive formula (3), substitute from (4) and (7) in (8) to obtain 
M8. 
® ge Ae) 
ved by noting that DOG Y 2 + La. Then, 


Formula (2) may be deri 
substituting in (9) Z ) 

2) /(М —1 
в _ (82-1) MS, 
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AN IMPROVED PROCEDURE FOR THE WHERRY-WINER 
METHOD FOR FACTORING LARGE NUMBERS OF ITEMS 


Leroy WoLINS 
IOWA STATE COLLEGE 
ented that differs from the previous one in that the 


liminated from the computations; thus some formu- 
d method is presented. 


A technique is pres 
use of variance terms is e 
las are simplified. A rationale for the improve 


the following formula is recommended for 


In a previous article [1] 
tes of factor loadings: 


successive use to secure estima 
ск” 2 

(һе £- 1) T hi 

ia Lo 


e: - п) 1099 2 


(0 rue 


r loading of an item. The difference between tix’ 
based on computations involving communalities 
the same correlation matrix but with unity 
deviation of a total score based 


In (1) rix is the facto 
and тұқ is that the latter is 
whereas the former is based оп 


entered in each diagonal cell. The standard 
on all of the items included in a cluster is ок’ » and the average standard 


deviation of the items included in cluster К is 0: - The number of items 1п- 
nd, of course, represents the number of ones In 


cluded in cluster К is пк’ 21 ‹ 4 
the diagonal of the matrix of item intercorrelations. ТЖ 
Ву making substitutions in (1) suggested by the relationships 


@ ве = ж Sr 


and 
ot =. 


(8) 
(1) becomes 


(4) rk = Ara) — "к 
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As in the previous method the correlations of items not in the cluster 
are converted to factor loadings by multiplying them by С. and 
(5) fn = СС, тке) 


but Бу the present method 


E 


p» Тік" 

(6) Ск, = =, 

The rationale for this modification is discussed in connection with the 
following identity: 

resides Mora] + в. 
ЕГЕ: „е Іі 
m ааа EE * 
where 
Tox? = the correlation of a particular item, а, with the cluster that 
contains it, 

Ter; = the correlation of a 


particular column or TOW of item intercorre- 
of item standard deviations, 


Ce; = the standard deviations of the item standard deviations 

Cray = the standard deviation of a particular row or column of item 
intercorrelations, 

б; = the mean standard deviation of the items,. 

74; = 


the mean of à partieular со 
"cio, = the correlation of the prod 
tions of two item Standard 


lumn or row of item intercorrelations, 


‘uct of all combinations and permuta- 
deviations with 


their respective item 

intercorrelations, 

Фе; = the standard deviation of the product of all combinations and 
permutations of two item standard deviations 

©, = the standard deviation of all item intercorrelations 


f; = 


The first terms t 


0 consider are Tra; and o,,,. With respect to manipulating 
the data so as to make these terms Zero, nothing ean be done. It is expected 
that the correlations within а correlation та 


of such a matrix will 

less than one, and if 

Will be quite small. 
The c,, and Teisi 


ations will certainly be 
the clustering was done well, these standard deviations 


terms can be manipulated. 1% is conceivable with the 
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use of electronie computors that the distribution of item responses to each 
item could be economically transformed to standard scores prior to com- 
puting the sums to obtain cluster scores. Later comments will point out that 
this is probably not justified. 

The Toara; and Гезереа terms will be zero on the average, theoretically, 


if tetrachories are used and, theoretically, will be positive on the average 


if phi coefficients are used, varying in size depending оп ae; and оз; . How- 
ever, even if chance were the only thing contributing to the variability of 
Танат AN таар з the user of the present technique would not wish to 
depend on any theoretical average value since the standard error of these 
coefficients would be quite large with the small number of observations which 
would occur in those problems requiring iteration. 1 

In the Wherry-Winer paper the assumption made was that Ta; 18 zero. 
It was also assumed, tacitly, that the bracketed terms m the denominator of 
(7) are the same for tetrachorics as for phis. Since both Ter. and с. will change 
as а result of using tetrachorics, this latter tacit assumption is not justified; 
however, since all three terms in both the denominator and the numerator 
will be less than one and probably closer to zero than to one, the product of 
the three bracketed terms will probably be very small. This appears to be the 
reason Wherry and Winer found close correspondence of their шошо 2 
results obtained through actually extracting à centroid. This E z len 
when one considers that if items included in а cluster vary in is я {у de 
much as from .2 to .8, о»; will be in the order of .02 and See; Will be even 


smaller. 3 
Thus, it is concluded on à жарату ae 
procedure is an improvement of the torn j vith n 
computational Per certainly when воне me ur n к 
to phis, it is difficult to determine if the present method 38 AIL dot Башар 
than the former one. One can вау with ош Ж ү » the bracketed 
difference between them on the average since the product о 


terms will be small in most cases. к | ч ME uh 
i i i d in [2], 15 clearly superior to bo 
A third method available, discussed in [ уз ДАН Ық AC 


: i lts of t AO 
methods discussed above, in Ul = зеді from a cluster. Ап objection to 


to those obtained from extracting а Ce x F a work. The exact 

the general use of this exact method is ұлы eta E TAES 

method should be used on those occasions йү aie 

and Қайс cs ms wishes to salvage e bog bcd. bi a du 
$ d , , 

that the exact method will salvage the с lues of the bracketed 


e val 
can be caused by а negative ri; 28 well as by larg 


terms. 4 

Another use of the exact method is сз е 
If items are responded to on а 1 to ? ВСВ 
exact method than to dichotomize. 


that the present computational 
one, certainly with respect to 


e items are not dichotomous. 
it may be easier to use the 
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The computational formula for the exact method is 


(ук, > бїк, — о) + ЇР; 
?; = 1 21? 
® 3 Ух [6:0 к, > б ік — о) + hi] 
where 
= Ж, 
(9) Тук = m 
REFERENCES 

[1] Wherry, R. J. and Winer, B. J. A method for factoring large numbers of items. Psycho- 

metrika, 1953, 18, 161-179. 
[2] Winer, B. J. Iterative factor analysis: its psychological and mathematical bases, Un- 

published doctoral dissertation, Ohio State Univ., 1952. 


Manuscript received 8/27/58 


Revised manuscript received 3/11/59 


| 
| 
| 


PSYCHOMETRIKA—VOL. 24, No. З 
БЕРТЕМВЕВ, 1959 


GENERATING VARIABLES WITH ARBITRARY PROPERTIES 


Рлоь J. HorFMAN 


UNIVERSITY OF OREGON 


There are occasions in psychological research where it is desirable to 
have available sets of variables with arbitrary intercorrelations. A quite 
simple procedure is described for generating pairs of such variables. 


There are a number of instances in research and teaching where it is 
desired to produce fictitious data that exhibit particular characteristics. 
Linear transformation of scores on an existing variable can easily produce a 
variable with any desired mean and standard deviation. In test construction, 
where item information is available, the items can be so selected as to yield 
a test of given difficulty and reliability [1]. It also has been shown [2] that a 
test can be constructed to yield a given weight in relation to a second test or 
& composite. 

This note describes a procedure whereby pairs of variables may be 
constructed from a table of random numbers so that their correlation will be 
of any given predetermined magnitude. The operations can be easily carried 
out in a few minutes on a desk calculator. 

Let X and 2 be random variables with the restriction r,, = 0. We wish 
to determine a distribution Y such that r., is of some arbitrary size. Let 


Ү, = x, + bZ; . In deviation score notation, 


X (x (as + bz) _ о + Моо. 


„= De No,9 (622 0,0 (42) 


i=l 


which, with r,, = 0 becomes 


о. 
1 = z 
Ф Tay [422] 
But 
(2) cha. eid Vei. 


Therefore, by substitution, 
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or, solving for b, 


bm Е 
= St) EN 


oS zy с; 


zy 


since V/1—r2, = k, = the coefficient of alienation, 
(8) b = е, 
from which the У; can be readily computed. 
The У distribution which results will have a mean, 
Ү-Х-м 
TABLE 1 


Two Normal Variables, X and 2. F= 


Z = 5.00, 
9x = 6; = 1.00, and туд = .00 


2 X 2 X 2 
5.966 1.219 5.42) 5.10 5.188 5.303 о), 1 
5.613 711. 6.26 14193 8.04, — 1.852 i Soe 2 166 
5.339 1.152 3.979 6.02, 1.910 — 6.363 1.989  ].187 
2.539 3.733 1.388 ^ 5.726 4.709 5.119 6.942 — 1794 
h.l, 5.851 5.160 — 6.72 3.712 5,60 


| 
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and a standard deviation, 


If it is desired that the Y distribution have some arbitrary mean, Y, , 
and standard deviation, с, , as well as an arbitrary correlation with X, the 
individual scores may be computed from the formula 


Y, = 74 (X, + 529 +C 


where c,, = the desired standard deviation, 
с, = the obtained standard deviation, Мо: о: 
@= = а (X + 2) 


and b is defined as in (3) above. 
In the special case, where 


x = Z, о: = в, = 1.00, and т, = 0, 
We find by substitution that 
b= вы PY 


XQ rs), ov = Мы. 


Table 1 contains, for № = 100, a sample pair of normal variables (X 
and Z) such that X = Z = 5.00, e. = e. = 1.00, and r.. = .00. Additional 


‚ Variables may be obtained from the author by request. 
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A NOTE ON THE TRYON-KAISER SOLUTION FOR THE 
COMMUNALITIES 


Henry Е. KAISER 
UNIVERSITY OF ILLINOIS 


. The Tryon-Kaiser solution for the communalities is reviewed. Numeri- 
cul investigation suggests that the procedure is applicable if and only if the 
correlation matrix has unique minimum rank communalities. This implies 
that this approach to the communality problem is not general enough to be 


of practical use. 


: In his recent review of the notion of communality from a cluster-analytic 
viewpoint, Tryon derives a formula "for the exact value of ^" ([6], eq. 21). 
This treatment is interesting theoretically because it does not explicitly 
consider the dimensionality of the common-factor space. It is interesting 
practically because—at least for one example—it succeeded in obtaining an 
exact solution for the communalities. 

- Simultaneously and essentially 
ditional factor-analytie point of view deve 
Approach. His treatment consists of a derivation 
equations 


independently, Kaiser [3] from a tra- 
loped what ultimately is the same 
and an attempt to use the 


НЕМ 1” (і-1,2,--,Ф, 


to provide an iterative solution for the communalities, where Aj is a trial 
value for the jth communality, # is a new (and hopefully improved) approxi- 
mation to a solution, # is the jth diagonal element of the inverse of the 
reduced correlation matrix with hj in the diagonal, and % is the number of * 
Observed variables. This formula is applied as an attempt to compute the 
Squared multiple correlation of test j on the remaining arbitrarily large number 
of tests in the hypothetical domain of content under consideration, a value 
a under very general conditions may be shown to equal the communality 
2]. 


This note reports results obtained when this method of solving for the 


H 243 М 
communalities was applied more extensively. In addition to Tryon’s successful 


example, exact communalities for three further matrices were easily solved, 
using Kaiser’s equations. These three examples, like Tryon’s, had the property 
that the number of common factors was less than half the number of tests, 
as the off-diagonal elements of the correlation matrices involved had been 
generated artificially by multiplying ап arbitrary factor matrix with this 
Property by its transpose. For a second group of examples—six correlation 
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matrices based on empirical data—this method did not yield communalities; 
iterative procedure failed to converge. 
d DOE de question is whether this method will succeed raton 
artificial correlation matrices and not for data from the real world. T » 
answer probably lies in а theorem of Albert's and in some results of Leder- 
mann. If r is the number of common factors, Albert [1] proved that when 
т < n/2, there exist unique communalities such that the resulting reduced 
correlation matrix has rank r. If it is postulated that the Tryon-Kaiser 
procedure will be applicable if the correlation matrix has unique minimum 
rank communalities, Albert’s theorem would account for the success with 
artificial matrices. On the other hand, empirical correlation matrices do not 


have unique minimum rank communalities. This follows from Ledermann [4]. 
He has shown that if 


т > $n --1— V8n+ 1), 


the communalities will not be unique. He has also shown that if r is to be less 
than $ (2n + 1 — V8n + 1), special conditions must hold exactly among 
the off-diagonal elements of the correlation matrix. Because sample corre- 
lation coefficients are continuous random variables, Ledermann's special 
conditions can only hold with probability zero, and consequently in practice, 
unique communalities may occur only with zero probability. 

By systematically varying r/n in constructing additional artificial 
correlation matrices, extensive further numerical investigation uniformly 
confirmed the hypothesis that the Tryon-Kaiser solution for the communalities 
will converge if and only if the correlation matrix under consideration has 
unique minimum rank communalities, Indeed, unique negative “communali- 
ties” from non-Gramian matrices (generated with imaginary factors) were 
easily found. Attempts to prove the hypothesis algebraically with methods 
described by Scarborough ([5], pp. 209 


-211) have not been successful. 
The difficulty with the Tryon-K: 


aiser solution is that it is incomplete. 
What is needed is a criterion for gel 


ecting among the inevitable multiple 
solutions for empirical correlation matrices. What this criterion might be 
seems a difficult scientific (not mathe 


matical or statistical) problem. It does 
not appear to have been explored systematically, 
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А NOTE ON THE USE OF TRIADS FOR PAIRED COMPARISONS 


В. E. ScHUCKER 


PURDUE UNIVERSITY 


. When sealing a large number of stimuli from comparative judgments, 
considerable savings in time and labor may be realized if stimuli are presented 
in triad form rather than in pairs. If, for № stimuli, the proper configuration 
of triads can be assembled so that all possible pairs appear once, the paired 
judgment matrix may be reproduced with one-third fewer judgments and 
two-thirds fewer presentations than would be required with complete pairing. 
А simple procedure is described for enumerating triad configurations for which 
N is an odd multiple of three. 


Applieation of the traditional method of paired comparisons rapidly 
becomes unwieldy as the number of stimuli increases beyond 20. However, 
the labor required in eliciting and analyzing large numbers of paired judg- 
ments has been greatly reduced with the advent of punched card procedures 
[3]. In addition, the burden on the subject has been eased through the develop- 
ment of a partial pairing technique [5, 6]. With partial pairing, the total 
number of pairs to be judged may be reduced any desired amount by pairing 
each stimulus with fewer than the М — 1 remaining stimuli. 

Where the paired comparison method is applied for the purpose of 
developing an interval scale, the investigator may desire that all stimuli be 
Sealed with equal accuracy. That is to say, it is required that each stimulus 
be compared equally often with all remaining stimuli. It is readily apparent 
that the partial pairing technique does not fulfill the requirements for balance 
With respect to the estimation of stimulus scale values. Hence, when large 
numbers of stimuli are to be sealed, an alternative to partial pairing is needed 
if scale values are to be estimated with equal accuracy and the volume of 
paired judgments kept within reasonable bounds. M. | 

One approach in reducing the total number of pairs in paired com- 
parisons has been through the use of triads. The format of the Kuder Prefer- 
ence Record is a familiar example of the grouping of stimuli in three’s rather 
than in pairs [4]. If for the triad A, B, C, judgments of the type “most” and 
"least" are obtained, with respect to the ordering of the objects along a 
psychological dimension, preferences for the pairs (A, В), (A, С) and (В, С) 
may be recovered. Similarly, if for N objects the proper configuration of 
triads be assembled, so that every possible pair appears exactly once, infor- 
mation for the complete, paired judgment matrix is obtained with only one- 
third as many triads and two-thirds as many judgments as would be required 
using pairs and traditional full pairing. For example, complete pairing for 
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an N of 57 requires 1596 pairs, while only 1596/3 = 532 appropriate triads 
"ni sa aired information. | 

Ж н of finding the triads needed for a given N in € 

that each element be paired once with every remaining element. The oe 

paper touches upon a geometric solution that produces for certain N’s a 

basic group of triads, from which the required configuration m | 

ated by cyclic permutation. Where the stimuli to be scaled are verbal in 


nature, the cycling procedure is readily adaptable to punched card equipment 
for the preparation of the triads and analysis of data. 


ay be enumer- 


Cyclic Enumeration of Configurations 


In general, a configuration consists of № elements arranged in b sets of k 
elements each, with each element occurring in r sets, and each pair of elements 
occurring together in a set exactly А times. The foll 


owing relationships must 
hold. 
r(k — 1) 
0 С ее 
(2) Nr = bk. 


For the case under consideration А = 1 and Ke = 20 Substituting these values 
in (1) and solving for г, yieldsr = (М — 1)/2. Thus a configuration of triads 
satisfying the required relationships is possible only when N is odd. In 
addition, it should be pointed out that the conditions specified in (1) and (2) 
are necessary but not sufficient for the existence of a configuration. Ha 
determined that a particular N satisfies the requirements, 
ling the triads of the configur 


ation still remains. 
Various methods for constructing configurations h 
by Cox [2]. Class \ = 1 configurations h 


in the literature and humerous solutions 


ving 
the task of assemb- 


ave been reviewed 
ave received considerable attention 


are available for the subclass k = 3, 
Where N is an odd multiple of three. Solutions for № of this form may be 
obtained by cycling basic groups of triads. Depending upon N, the 
enumeration of the com 


plete configuration from the basi 
l-step cycles, 2-step cycles, 3-step cycles, ог combinati 
example of enumeration by 1-step cycles is provided by the со 


nfiguration for 
N = 9. From equations (1) and (2) it is seen that r 


= 4.0 = 12. The configura- 
tion is completed by setting N = 9. 
Rep I Rep II Rep III Rep IV 
row set set set set 
м (45 N g уаз Т (ш Xa 
2” (2272 878 CB) 3 aby 4 (8 4 5 9 (11) 5 6 2 
9 (9)76 724 (0) X. EB (9) 8 1 6 (12) 4 6 7 


Тһе triads are grouped into 


complete replications of 
will be observed that the rem 


p the 9 elements. 1 
211118 7" — 1 triads in any 


row may be obtained 
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by а succession of one-step cyclic permutations on the elements of the triad 
appearing in the same row in Rep I. For example, the addition of 1 to each 
element in triad (3) gives triad (6). Similarly, by cycling the elements of 
triad (6), triad (9) is obtained, and so forth. Note that the second element 
of each of the row-1 triads is not cycled and that for all triads one re-cycles to 
1 on the next step when an element equals № — 1. Since the complete con- 
figuration can be generated from Rep I, the sets of Rep I may be arbitrarily 
designated as solution triads. 

А second configuration is shown to illustrate enumeration by two-step 
cycles. For this example, № = 15, г = 7, and b = 35. 


Rep I Rep II Rep III Rep IV 
set set set set 
(1) IN 8 (6) 3 М 10 (1) S N 12 (16) TEN 14 
(8%) £ вт (Лу б $79 (12) :&' 00 (17) GEO аЗ 
(3) 13 14 9 f(s: 1 Bd (18) % 4,48 (18): 5, "6 i 
4) зви (0. & E48 бт 
(5) 10 12 2 G0) 12 1a‘ и (16) 344 9-8 (20). гома 
Rep V Rep VI Rep VII 
set set set 
CD ай @ сб n NW 4. аи 
(22) 12 13 1 (95. AL yL^3 (922) 2 8.270 
(28) 7 в а (93). 9 10. (09,1 107 
UD Им Б (200 18 2 7 (34) f 2 9 
(25) 4 6 10 (30) 6 812 (35) 8 10 14 


Setting № = 15, the configuration satisfies equations (1) and (2). As before, 
the r — 1 remaining triads in any row can be generated starting with the 
corresponding triad of Rep I. For this example, however, cycling proceeds 
Y 2-step increments. Thus, by adding 2 to each element of triad (2) one 
obtains triad (7), which gives rise in succession to triads (12), (17), (22), (27) 
and (32). Note that a re-cycling to 1 or 2 occurs on the next step for elements 
Which equal N — 2 or N — 1. a 
Ball discusses variations on a geometric method for obtaining solution 
triads in configurations for which М is an odd multiple of three ([1], ch. 10). 
Essentially, the method involves the use of a circle with inscribed triangles, 
the points of which represent triad elements. The triangles of one replication 
are determined empirically, with the restriction that, when rotated within 
the circle, they generate the remaining replications of the configuration. 
Ball gives solution triads for № = 9, 15, 21, 27, 39, 39, 45, 51, 57, 63, 69, 75, 
51, 87, 93 and 99. 


Machine Cycling of Triads 
n connection with the con- 


The cycling procedures demonstrated 1 
" arious card punch 


figurations for A's of 9 and 15 are readily adaptable to v 
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calculators, which can be utilized for preparing the triads on IBM cards as 
well as for generating configurations. Judgments can be recorded directly on 
the cards and the analysis facilitated by using tabulating equipment. ‘The 
program requirements for enumerating triad configurations are relatively 
straightforward and may be deduced directly from the two illustrative 
examples.* Either modular arithmetic or the conventional arithmetic shown 
may be employed, although use of the former results in а, shorter, more general 
program capable of handling cyclic steps of any magnitude. 
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DONALD DAVIDSON, PATRICK SuPPEs, AND SIDNEY SIEGEL, Decision Making, An Ezperi- 
mental Approach. Stanford: Stanford University Press, 1957. Pp. 121. 


А The basic problem which the authors have set out to tackle in the studies reported 
in this book is the separation of the effects of psychological probability and utility in de- 
cision making. They have considered this problem both in its theoretical framework and 
in the experimental verification of the theories set forth. The first chapter gives an intro- 
ductory discussion of the problems of empirical interpretation of theories of decision making 
under uncertainty. The second chapter fills almost half of the book and deals with the 
basic model proposed by the authors. The third chapter reports an experiment which was 
designed to measure the cardinal utility of nonmonetary outcomes and to use the computed 
utilities to predict further choices. Two models were compared, one a linear programming 
model and the other an ordinal model based on straightforward comparisons. The linear 
programming model turned out to be considerably superior to the other and both were 
much superior to a random guessing method; moreover, if thresholds are ignored in order 
to obtain a larger number of predictions the accuracy remains significantly better than 
chance. The fourth chapter considers the problem of formulating utilities for incomparable 
outcomes; in contrast with the two preceding chapters the considerations here are entirely 
axiomatic in character. T 

The second chapter offers an explicit theory for the explanation of individual decision 
making under conditions of risk, and reports an experiment designed to test the theory 
in certain limited situations. The first step in the theory is to construct an event which 
has a psychological probability of one-half. Next, a set of six outcomes is constructed so 
as to be equally spaced in utility, and from these a utility function is constructed which 
is adequate to account for a certain class of preference and indifference relations. The 
experimental results lead the authors to conclude among other things that (1) the theory 
Provides a practical approach to the problem of resolution of utility and psychological 
Probability in situations involving risk; (2) under suitably controlled conditions certain 
People make choices among risky alternatives as though they were attempting to maxi- 
mize expected utility, and (3) for such persons it is possible to construct a utility function 
Unique up to a linear transformation. The poin 


t of departure for this model was the work 
of Mosteller and Nogee (An experimental measurement of utility, Journal of Political 
ions with other previou: 


gala 1951, 59, 371-104); connecti s work are traced and a short 
ut well-chosen bibliography is included. ы К m 
In my opinion Ws book will take a prominent place in the literature of decision 
making, but it is also clear that it does not represent the final word on any of ү pus 
considered. The expository level of the book is excellent; the prospective reader should be 
prepared for a reasonable amount of mathematical development of the axiom-definition- 
th Ў 
Poron ра В. M. THRALL 


University of Michigan 


Warren S. Tonaknsox. Theory and Methods of Scaling. New York: John Wiley and Sons, 


1958. Pp. xiii + 460. Я 3 

In 1950 the Social Science Research Council appointed a не 
"Theory and Methods to review the status of scaling procedures in h cs un and zolid 
committee came to the inevitable conclusion that a good € e ape ER Ра 
work on scaling procedures was necessary- In 1951, Warren "im scaling procedures. 
Associate of the Council, undertook the preparation Domo dE 
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After seven years, two of which constituted а long lost weekend in the Navy, the mono- 
graph had become a book and was published. 


The result of Torgerson's and the Committee's efforts is a book which will be of 


scientists. It is an excellent summary of 
in the social Sciences, the sort of book of 
which there is a very real shortage in psychology and the social sciences in general. It was 
probably cannot be used as such. While 
cal workers, it does seem possible to use 
atical background is assumed on the part 
used than was anticipated. 

е due more to the state of the art than to 
or. Torgerson provides an excellent organi- 
are in fact used; the various methods are covered 


to use a particular method after а thorough reading 
~ of the appropriate section of the bo: <. Пе ех 1 


ical treatment of the 
at the abstract mathe- 
cting data and treating 
in other words, a happy 


ry and organizing chapters—in this 


the nature of measurement, types of 


psychology 


ponse approach, form the basis fo 
of the book. The term response for th 


the second, in which г 
The next seven chapters deal with the judgment me: 
fractionation, and equisec 


n on the part of the subje 
subjective continuum under considerati 


native, or differential sensitivity ethods which are based pj. 
on the Thurstone models. The last the response methods, 
One chapter is concerned primarily with the Guttman techniques, one js concerned with 
Lazarsfeld's latent-structure model, and the 1 


" ] ast is concerned with the techniques de- 
veloped by Coombs and his students for dealin 


rs are de- 
e methods 


ty methods, and treats 
hysical methods. In the rest of the book if 
e chapter, however, 
and it would һауе 
subject at all, 

ess with what "'orgerson 
n psychology today. For 
ata matrix which аге used 
e nature of the data. He 
вага to both the stimulus 
а more fundamental look 


on the basis of th 
de with re, 
$ provides 


і 
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at the over-all picture than is customary. However, I found a desire to go back a step 
further, and to note that all Базе sets of data involve three variables—stimulus, subject, 
and response—and that all of them have certain relations to the underlying true conti- 
nuum. Just as each stimulus, or each subject, can be located on the continuum, so can 
each response—and there ean be interactions between all three variables. It is not really 
necessary to assume anything fixed by the response, even а comparative response, and 
a truly general model for measurement would inelude solutions for the response as well 
as for stimuli and subjects. It is usual, of course, to have stimuli and subjects be orthogonal 
in experiments, while neither will normally be orthogonal to the response variable. It is 
quite possible, however, to give each subject a response and then tell him to find the stim- 
ulus which satisfies this response—just as is done with some fractionation procedures and 
with the equisection procedure. Any truly fundamental model for measurement must be 
able to deal with the relations between all three variables and the underlying continuum. 

Actually, of course, we run into the reality of the lack of degrees of freedom for 
complete solutions, as Torgerson so often points out. And in fact the practical solutions 
with which we must deal often are such that an equivalent solution could have been ob- 
tained with simplifying assumptions other than those actually made. Using assumptions 
about the values or distributions of any of the three basic variables, we can solve for 
values of the other one or two. A change in the variables about which the assumptions 
are made will change the solutions as well. In other words, we cannot create more knowl- 
edge than the data give us; we can simply assign the knowledge to different variables. 

In this frame of mind, I have one last reaction to report. It is that we have more 
sophistication about the nature of mathematical models of measurement than we do about 
experimental techniques for validating the models, Elegant scales can be constructed, but 


only after we have made enough assumptions to reduce the number of parameters to the 
number of available independent observations. Any verification of the assumptions re- 
and experimental techniques must be 


quires the availability of more degrees of freedom, 1 t 
devised which provide these degrees of freedom in a form appropriate to the assumptions 
es. made. The mathematical models tell us only what can be so; better experimental 
echniques are necessary to tell us whether it 2s so. , 
"These imis, ai 2 intended as criticisms of the book, but rather as compliments 
to it. The book presents the whole range of material in sufficiently compact form that 
Опе is forced to try to get an overview. This fact, plus the over-all excellence of the pre- 
Sentation, will stimulate new and good research. I am tempted to say—and so I will say— 


that this book is a milestone. Жой. в. Gaetan 


The Johns Hopkins University 
D. A. 5. Fraser. Nonparametric Methods in Statistics. New York: John Wiley and Sons, 
1957. Pp. x + 299. 


, Vonparametric Methods in Statistics ( 
NMS consists of two parts: an introduction 
tradition in statistical inference (Chapters 
ments to nonparametric statistics (Chapter: 


survey of limi ful in nonparam! 3 s 
"The a pagi E, ache n eases for comfortably reading NMS is а year’s 


course in function theory. With less than advanced зеш [у a qs т 
the definitions and theorems are hardly intelligible. Mood (жөн с 
Statistics, MeGraw-Hill, 1950) or preferably Cramér (Майетанса with: 
Princeton University Press, 1946) are reasonable prerequisites in statis $ 


nced work in statistical theory. 
to recent developments in the Neyman-Pearson 
1 and 2) and an application of these develop- 
s 3, 4, 5, and 7). In addition, Chapter 6 is a 


etrie theory. 


NMS) is ап adva 
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out this background, NMS can possibly be used as a reference book f. 
book (Nonparametric Statistics, McGraw-Hill, 1956), 


of theory and mathematical content. 

Besides the standard Nelnan-Pearson optimum properties (e.g, most powerful, 
unbiased, and consistent), sufficiency, invariance, and completeness аге stressed. These 
ideas are developed extensively and used in finding good nónparametric procedures— 
tests of hypotheses, point estimates, and tolerance intervals. Sufficiency and invariance 
have a strong intuitive appeal as criteria for optimality. Completeness is a mathematical 
condition that is useful when available. To illustrate these ideas consider the following 
problem. It is assumed that X; , --- ‚Хь, Yi, +++, У, are mutually independent random 
variables. Assume all of the X's have a common distribution and all of the Y's have a 
common distribution (not necessarily the same as that of the X's). How should one esti- 
mate Pr (X; < У;)? (Pr (X, < У;) appears in the study of the Wilcoxon two-sample 
procedure.) 

It is clear that the (temporal) order in which the observations are made is irrelevant 
and attention can be restricted to Хау, -- „Хоу, Yay, >, Yo, where Хау is the 
smallest of Xi, --- , Xm ; Xe is the second smallest, ete, In short, the order statistics 
form a sufficient statistic for the problem. The parameter of interest, Pr (X; < X i), will 
have the same value whether the origi d or whether any mono- 
tone increasing function (e.g., le) is used. Therefore it 
is reasonable to restrict attenti 


or the theory. Siegel’s 
at the other extreme, is nearly devoid 


invariance. For this problem, i estimator must be a function 
of the ranks. Completeness impli i i i 


The approach of the above paragraph i 
experiments which arise in practice, e.g., 
about the location parameter of a distri 
samples come from the same distribution 
from distributions differing in location o 
dependence in a bivariate distribution fi 
mating correlation; and (d) constructing 
data. 

The orientation towards Neyman- 
variance, etc.) explains the lack of emph: 
the theoretical orientation, many math. 
hand, applied examples and tables of di 


› divided by mn. 

imating and testing 
т à random sample 
ull hypothesis that c 
at the c samples come 
about the amount of 
, testing for independence and esti- 
tolerance sets from univariate and multivariate 
Pearson theory and linear models (analysis of 
asis on tests of goodness of fit, In keeping with 
ematical examples are given, and, on the other 
istributions are not given. 


I. Біснанр SAVAGE 
University of Minnesota Фа 


Рипар J. McCarrny, Introduction to Statistical Reasoning. New York: M c-Hi 
1957. Pp. xiii + 402. Mice EB, 
The author states his aims clearly 
matical course in statistics in which the i 
to statistical reasoning. .. 


in the Preface: ч 


Шек. - ай о. ео 
—À N ыз. о. a 
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to improve his insight into the problems of research methodology in the social sciences. .. . 
This selection of illustrative material from the social sciences has also influenced to some 
extent the topics discussed in the book.” ы 

I agree with these aims and judge that the author has met them very well indeed; 
hence, I recommend this excellent book for a one-semester, nonmathematical, introduc- 
tory statistical course in the social sciences. It excels in the choice of exercises and illus- 
trations drawn from important research publications in the social sciences, has many 
good examples and fine figures, tables, and charts to illustrate the important problems. 

How does this book differ from some of its better competitors? First, it emphasizes, 
as the title states, statistical reasoning and inference rather than statistical techniques 
and manipulation. Rather than trying to provide a reference book on a large variety of 
Statistical techniques, the author concentrates on a thorough presentation of the nature 
of statistical inference. He does this briefly in two chapters which contain the two most 
useful statistical methods for social scientists: Chapter 8, The Binomial Probability Model 
and Statistical Inference; and Chapter 9, Drawing Statistical Inferences from the Arith- 
metic Mean of a Large Sample. 


Second, the statistical problems and examples are not alienated from their origins 


and their destinations. The problems and examples are established firmly in the substan- 
ч problems from which they arise, and in the problems of collection and processing 
which precede the data. The relation of sample to population is often and well developed. 

е meaning of statistical tools and of statistical inference is constantly emphasized; 
that Statistics and probability statements are guides to action and to decisions is the 
Spirit that pervades the presentation (although there is no formal presentation of statisti- 
cal decision function theory). Chapter 2, The Components of Statistical Investigation, 


presents this approach early and well. 

ана Кы the writing is rigorous and р 

aid oes not require a mathematical backgrou 
careful reading. The style is clear, precise, 2 


through the hard thinking that rigorous statistical 
vith the necessary 


recise. The presentation is not mathematical 
nd, but it does demand close attention 
nd to the point, and will aid the student 
1 inference demands. ка 
. Chapt 7 vell w introductory topics of distribu- 
tions, their алам» К the elements of probability. Chapter 10 gives in 
ә. Pages an excellent, lucid, and penetrating presentation of Elements of Sample Design. 
ach of the last two chapters presents an important technique for social scientists: Сһйр- 
Ж 11, Chi-Square Procedures for Qualitative Data, and Chapter 12, The Linear Associa- 
9n of Two itati iables. р 5 
Some к fe to add one or two additional пинк и ойе 
Ша course; for economists, perhaps time series and indexes; for psycho 0р18:8, perhaps 
i experimental design. The 


е difference of two correlated means and the elements of > 
Student will have js read his assignments thoroughly and sometimes repeatedly. The 


instructor will have to explain the finer points at greater |. БЫ репер ыу а not 
too much to ask of a scientific subject in а scientific age. I think this 1s а E Я 
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AN APPROACH TO MENTAL TEST THEORY* 
FnzpERIC M. Lorpt 


EDUCATIONAL TESTING SERVICE 


It currently seems to me that the heart of mental test theory is the 
concept of true score. 

The trouble starts as soon as it is realize 
obtained by a particular individual might just as well have been some 
numerical value other than the one actually observed. The individual ex- 
aminee might have guessed differently, might have been less nervous, or 
might have slept better the night before. The testing conditions might have 
been different—lighting, facilities for use of paper and pencil, presence or 
absence of distraction. Finally, any one of many different but equally appropri- 
ate psychological tests might have been constructed and administered. 

Mental test theory must deal with all these kinds of disturbing influences. 
However, we are not really interested in each of the different test scores that 
an examinee might obtain under all sorts of conditions. We are interested in 
Something approximated by these scores. This something may be called the 


true score on the test. 
Many definitions of true sc 


d that the actual test score 


ore are possible. Each is open to serious 


objections. The hope is that when a particular definition of true score is 
used, this definition will lead to useful approximate statements about the 
even though this is not adequately 


true value that we are interested in, 


efined., 

Let us assume that the error of measurement, defined аз the difference 
between true score and observed score, has an expected value of zero in all 
circumstances. This is the only assumption needed in order to obtain accurate 
estimates of the mean true score for а group of examinees, provided that 
the number of examinees is fairly large. I mention this in order to point out 
that some useful and meaningful results about true score can be obtained 


Without makin: i tionable assumptions. 
са lot of highly questi 
What further assumptions or definitions can profitably be M a x 
additional inferences about true scores? I plan 2 consider three ашегеп 
types of a; i ill first describe briefly. 
pproaches, which I will firs d 
Perhaps the simplest and most straightforward approach is to assume 
Sept надан address delivered to the Psychometric Society, Cincinnati, Ohio, 
ir cs i der Contracts Nonr-2214 (00) 
Much of th i d here was carried out under 
and -2782(00) with ‘the Sten at Naval Research, Department of the Navy- 
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that one can obtain independently at least two, and preferably more, test 
scores such that the scores for any single examinee differ only because of 
errors of measurement. From a practical point of view, it is reasonable to 
make such an assumption about the carefully matched split-halves of a test, 
or even split-thirds, fourths, fifths, ete., provided the original test contains 
a sufficient number of items. 

Instead of this, a second approach goes back to the separate items of 
which the test is composed. According to one line of reasoning, a very large 
pool of items is defined in terms of the properties of the items actually at 
hand; true score is then defined in terms of the hypothetical performance 
of the individual examinee on this very large pool of items, assuming no 
practice effect, etc. A closely related model simply assumes the items at 
hand are a random sample drawn from a very large pool. 

Finally, a somewhat different approach is to make some restrictive 
assumptions about the frequency distribution of the errors of measurement. 
The assumption that this distribution is normal with constant variance is 
adequate for many common situations, A more plausible model assumes that 
the distribution of the errors of measurement is binomial, 

It seems to me that all three of these approaches to the problem of 
making inferences about true scores are very useful. They lead to different 
models for a theory of test scores, Five of these will be discussed in some 


detail, both practical and theoretical, skipping in part or in whole those 
results that are already well known. 


Before proceeding, it will be w 
between, on the one hand, the true 
a variable variously called the com: 
underlying the test," 
is that the true score 


1 и 3 m possible score such as 
the total number of items in the test; whereas the 


"ability underlying the 
test" is conveniently taken as having a possible range from — © to + о, 


A more basic distinction is that nonparallel tests of the same psycho- 
logical dimension will each necessarily have a diffe 
just as each has a different observed-scor 
tests of the same psychological dimensio; 
ability metric, once this 
the main virtue of the | 
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tests of the same psychological dimension have a perfect curvilinear corre- 
lation—one is simply a nonlinear transformation of the other. 

The advantages of the latent continuum are achieved at the cost of the 
more restrictive assumptions required for a stronger mathematical model. 
If we do not need to compare nonparallel tests of the same psychological 
dimension (in some cases even when we do), the stronger assumptions of the 
latent continuum model ean be avoided and a consideration of true scores 
will be quite adequate. 

Before proceeding with a discussion of the results obtainable from 
various mathematical models for true score, it may be pointed out that 
certain of these results may be applicable to the most varied types of problems 
in all scientific areas—typically, whenever the problem involves a large 
number of observed values, each of which contains a sizable unbiased error 
of measurement. A familiar example in the area of psychometries is a problem 
where one has a large number of item-test correlations or item-criterion 
correlations, Since each of these is subject to a large sampling error, various 
questions arise. One would like to know the frequency distribution of the 
“true” item indices—the values that would be found if sampling errors 
could be eliminated, One would like to have an estimate of the true corre- 
lation for each individual item. One would like to know the effect of discarding 
all items whose observed correlations are below some fixed value. Given the 
Sampling distribution of the individual correlation coefficient, these questions 
and related ones could be answered, at least approximately, by using certain 


of the approaches to be discussed here. 
Matched-Forms Model 


The matched-forms model for the relation between t 
observed score involves only two, very weak assumptions: (i) the expected 


value of the error of measurement is always zero (this statement applies 
to any individual or group of individuals selected in a fashion that is ex- 
Perimentally independent of the scores containing the errors in question), 
Gi) the true score of each individual is assumed to be the same on each of 
the test forms. Given these assumptions, no further definition of true score 
ls needed. The matched forms themselves provide the necessary definition 


of their tru ге. | 

Tinder thes aeania given k matched forms of a test, ваще 
can be obtained for all the moments of the frequency distribution of true 
Scores up through the kth order. From three matched forms, ag ea 
of the skewness of the true-score ath be obtained; from four 
Matched forms, an estimate of kurtosis; and so fort. , 

The kind “of ae аго under this model may be illustrated E 
the simple relationship in (1), which holds exactly in the population o 
examinees and approximately in large samples of examinees. 


irue score and 
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(1) SX. Хе Ж; 


1.е., the kth moment of the true scores is equal to the product of the examinee’s 
raw scores on k matched forms of the test, this product being averaged over 
all examinees. 

The moments of the frequency distribution of the errors of measurement 
can also be estimated, as can the multivariate moments involving both true 
score and error of measurement. From available formulas, all these estimates 
will be unbiased in random samples of examinees, even when the number 
of examinees is small, and regardless of the shape of the frequency distri- 
bution sampled. 

What can be done with these estimated moments? Methods for setting 
bounds on a distribution function from its first Ё moments are available 
[1, 12, 22, 28, 37]. Let me quote from М. С. Kendall ([13], p. 83): “For all 
ordinary purposes, therefore, a knowledge of the moments, when they all 
exist, is equivalent to a knowledge of the distribution function: equivalent, 


that is, in the sense that it should be possible theoretically to exhibit all the 
properties of the distribution in terms of the moments ... z 


“If now two distributions have moments u 
must have the same least-squares approximation [by a polynomial of degree 
n] +++ thus distributions which have a finite number of lower moments in 
common will, in a sense, be approximations one to another. We shall en- 
counter many cases where, although we cannot determine a distribution 
funetion explicitly, we may ascertain its moments at least up to some order; 
and hence we shall be able to approximate to the distribution by finding 
another distribution of known form which has the same lower moments. In 
practice, approximations of this kind often turn out to be remarkably good, 
even when only the first three or four moments are equated.” 

In practice, a Pearson curve having the same first four moments as 
the true scores may be used to approximate the true-score distribution. 
The Charlier and Edgeworth series are available when more than four 
moments are known. Other asymptotic approximations are summarized in 
а recent article by Wallace [34]. 

The mathematical derivations for the matched-forms true 
and the necessary formulas for estimatin 
appear in [20]. A discussion of the implic 
in [19]. In order to avoid undue repetition here 


p to order n equal they 


à Әп appropriate later point. 
The assumption that has been made here of strict parallelism between 
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the test forms сап be relaxed if one is willing to assume that the true scores 
on all the forms are linearly related to each other. With this assumption, 
one has the prototypical problem of linear structural relationship. Selected 
references in this very relevant area are [2, 9, 10, 11, 14, 15, 21, 24, 26, 33, 
34, 35, 36]. 


Rationally Equivalent Forms Model 


The fact that the matched-forms model is conveniently applied in 
practice to split-thirds, split-fourths, or split-fifths of a test suggests that 


there should be a true-score model that bears the same relationship to the 


matched-forms model as the Kuder-Richardson reliability coefficient bears 
lities. Such a method can easily 


to the split-half method of computing reliabi 
be formulated. It may be called the rationally equivalent forms model. 

For present purposes, the Kuder-Richardson approach may be thought 
of as follows. Imagine a large number of hypothetical forms of the test, 
a rigorously parallel to each other and to the actual test at hand. (By 

rigorously parallel” is meant that in any sufficiently large preselected 

group of examinees, the group statistics computed from the test scores 
will be the same, no matter which form of the test is used.) The crux of the 
Kuder-Richardson approach is to estimate the correlation between any two 
parallel forms of the test from item statistics computed for the single form 
actually available. In order to achieve this, one further assumption is needed, 
which, it seems to me, cannot be classified as part of the assumption of 
rational equivalence, as some writers have done. The algebraic derivation 
turns up with a term representing the covariance between item 2 in the test 
actually at hand and the corresponding item in the hypothetical test. Some 
assumption is required. The usual assumption is, of course, that the average 
value of this unknown covariance is the same as the average interitem 
covariance for all the items in the test at hand. р k 

With this assumption, the correlation between rationally equivalent 
forms of the test can now be estimated. Since this correlation 15 à reliability 
coefficient, it can be used to estimate the variance of true scores and the 
variance of the errors of measurement. It is implicit in all this that true 
score on the test at hand is to be defined as the average score that would be 
obtained on a very large number of rationally equivalent forms. It is only 
a short step from this line of reasoning to point out that the item statistics 
оп the one test actually at hand can be used not only to estimate the variance 
but also the higher moments of the true-score distribution. 

Let Хы be the score of exami 


nee а on test 1. Suppose that this score is 
the sum of the examinee's scores on 


n items, so that 


(2) ФТ 22 Шаға» 


ізі 
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where Zuia is the score of examinee a on the ith item in test и. The Bind 
results are obtained by first substituting (2) into (1). For example, if there 
are three parallel tests, then 


(3) УВ = Уз > P x Тао зга). 


g-1 hel i 


Application of the line of reasoning just outlined now transforms (3) into 


3 1 1 
(4) Y Les "а-а Оу У ати, . 


orhi 


F "more, quantities 
like those on the right of (4) can be expressed as a linear function of the 
moments of the observed-score distribution, to be denoted by т’. Thus 
(4) ean be written 


lo sw. 
(5) vës 
Under this mathe 


large sample of ех. 
estimation. Once 


1 
na = ia — эў (94 — 314 + 2mpj. 


r stics in the tests actually at hand. I am 
not completely happy with this model. rationally equivalent, 
s will usually be more 
. This is certainly not 
The model to be discussed nex 


premises, which, however, lead to formul 
illustrated by (4). 


m à somewhat different set of 
as not quite identical to those 


The Ii tem-Sampling Model 

This is the same model that was 
model [16]. In this model, it is assum 
can be considere 


previously called the matrix- 


sampling 
ed that the items in the test 


at hand 


OE ———— M 
а —————— à 


- 
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fieiently large so that we are willing to neglect quantities of order 1/N, 
the observable quantity on the right-hand side of equation (4) is no longer 
equal to the third true-score moment but is an unbiased estimate of it in 
random samples of test items. The accuraey of this estimate depends on т, 
the number of items in the test at hand. 

‘There is another important difference between the results obtained under 
the item-sampling model and those obtained under the rationally equivalent 
forms model. Under the rationally equivalent forms model, if we wish to 
know the numerical value of a product or power of true-score moments, it is 
only necessary to multiply together the numerical values computed for the 
moments in question. Under the item-sampling model, this is not sufficient. 
A produet of unbiased estimates of moments is not the same as an unbiased 


estimate of the product of moments. Special formulas must be derived to 
provide unbiased estimates of the various products called for by different 
ble [16, 17] so that unbiased 


formulas. The necessary formulas are availal 
estimates can be computed for any true-score moment about the origin or 
about the mean up through the fourth order. 

Also, formulas are presented for the bivariate moments between true 
Score and observed score. This makes possible the estimation of the regression 
of true score on observed score, even when this is curvilinear. By this method, 
therefore, one can, at least in theory, predict the true score of a given examinee 
from his observed score without assuming a linear regression. 

A useful result is obtained when linear regression is assumed. The formula 
for the regression coefficient in this case turns out to be very similar to the 
Kuder-Richardson formula-21 reliability coefficient. In fact, it differs from 


this coefficient only by quantities of order 77. . . 
Now, a little algebra makes it apparent that this regression coefficient 
Should in fact also be a reliability coefficient. In fact, there seems to be 
more intuitive meaning if one defines reliability as the regression of true 
Score on observed score rather than as a ratio of two variances. I was at first 
Somewhat puzzled, therefore, to find discrepancies between the regression 
Coefficient obtained under this model and the Kuder-Richardson formula-21 
reliability. The answer to this puzzle has been independently provided by 
Vageswari Rajaratnam, working with Lee Cronbach and, more recently, 
but probably not completely independently, by myself. Rajaratnam defines 
reliability in terms of the ratio of two variances and then derives a small- 
Sample formula for estimating reliability in the type of situation where the 
ltem-sampling model holds. When the number of examinees, N, is large, аз 
explicitly assumed in my formula, but not in hers, the two formulas are the 


same. 

; Equation (6) gives one for 
reliability coefficient, which is а 
on observed score, 


mple formula for this new 


m of the large-sa | 
e regression of true score 


lso the slope of thi 
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=... i 

(6) dud тесті 

where р is the average item difficulty, g is 1 — Ӯ, si is the variance of pro- 
portion-correct observed scores, and 8; is the variance of item difficulty. 

The relationship of this reliability coefficient to K-R (21) is not very 


apparent when the formula is presented in this form. The difference between 
the two coefficients can be shown to be 


Я 281-0 (1. 
(7) Ba (п — 1): ~ o(4). 


As long as the test has more than zero reliability, the difference between 
the two coefficients is actually a quantity of order тг, since the quantity 
1 — pin the numerator is itself a quantity of order n7}, 

When the number of examinees is large, the sampling fluctuations in р 
or 75; , due to sampling of items, are of the order of n~? *, Thus the difference 
between р and ra, is a half order of magnitude smaller than the sampling 
fluctuations in either. The difference between them will be 


negligible in typical 
testing situations. This comparison differs from that between the K-R (20) 


er л! and is thus 


; the small sample version 
from a logical point of view 
compared to their sampling 


seems preferable to Tai 
ence is usually small 


In working with va 


rious mathematical m 
disturbingly diffieult to 


find ways of checki 


to bring forward two practical applicatio 
these as yet unpublished studies, the true-score model is used to predict the 
moments of observed-score distributions, These latter distributions are 
subsequently obtained and the actual mome: 

values, 


In the first study, the problem is to 


\ 
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tribution. If the test is made longer and longer, however, the observed- 
score distribution will approach the symmetry of the true-score distribution. 
In this case, therefore, lengthening the test decreases the skewness of the 
observed-score distribution. 

As an example of the opposite situation, consider a case where the group 
of examinees tested has a somewhat skewed true-score distribution in the 
narrow range between .45 and .55. If the test is short and unreliable and is 
composed of 50-percent difficulty items, the symmetrical distributions of 
the errors of measurement will almost completely swamp the skewness of 
so that the observed-score distribution on a 
with observed proportion-correct 
lengthening the test 


the true-score distribution, 
short test will appear nearly symmetrical 
scores, possibly ranging from zero to one. In this case, 
will increase the skewness of the observed-score distribution. 

What is needed is a formula for determining from the available item 
statistics and score statistics on a single form of the test which of these 
things will happen—whether the skewness will decrease or increase as the 
test is lengthened. Similar questions, of course, may be asked for the kurtosis 
of the distribution. То 

How is it that true-score theory can be of use here? The reasoning is 
simply as follows. It is given that the longer test is parallel to the shorter 
test, except for length, i.e., the proportion-correct true score for any examinee 
will be the same on either test. Formulas are available for unbiased estimates 
of true-score moments in terms of item statistics and score statistics on the 
short form of the test. Such formulas are also available in terms of statistics 
on the long form of the test. We have, in principle, two unbiased estimates 
of each moment of the true-score distribution, one from each form. If these 
two unbiased estimates are equated, a set of formulas expressing statistics 
on the longer test in terms of statistics on the shorter test is obtained. For 
example, if the long-form statistics are distinguished by a prime, (4) leads to 


n’ (n! 20 (n -2) 
(8) M тұаСһыа ita 
"(т — 1)(% — 2) 222 ез 
n(n—-1)(n-2 


1 
— wy WXosUhaUia + 
nn = 1) — 2) AM ^ 


Since a moment of the longer test is a linear function of quantities like -— 
on the left of (8), the moments on the longer test can be id from the 
item statistics on the shorter [17]. Аз already mentioned, alge! 22 и 
lation makes the computations involved in equations like (8) = EN is 
Procedure has been applied AW e Y c MEINE etween 
redi as found for Ж. 
à EU и" was to predict the bivariate сораи 
of the scatterplot between two parallel forms of & test from the item statistics 
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of either form alone. Formulas for this have been developed [17] by a line of 
reasoning exactly parallel to that just described. These bivariate moments 
determine the shape of the scatterplot and, in particular, the shape of the 
regression of one form on the other (this regression need not be strictly linear 
even though the forms are parallel*). This procedure was also applied em- 


pirically and satisfactory agreement was found between predieted and 
actual values. 


Gaussian Errors of Measurement 


А somewhat different perspective emerges if one is willing to make more 
specifie assumptions about the frequency distribution of the errors of measure- 
ment. It is frequently an adequate approximation to assume that the errors 
of measurement are normally distributed with zero mean and constant, 
experimentally determinable variance. For convenience, this true-score 
model will be referred to as the Gaussian error model. Some extremely 
interesting and useful results can be deduc 
which I am sure have never been a 


the observed-score distribution. 


In my recent work I have been sear 
true scores without computing moments. 
the use of moments has certain 


y four moments of a distribution 
Js no о be at this is a particularly efficient 
method of approximating the distribution. The use of Pearson curves is 
probably often effective in practice, but i i 


guaranteed. It is well known that the Charlier series and the Edgeworth 
series may give poor fits to skewed distributions w 


Before proceeding, 


it is appropriate to ask whe 
distribution of observed 


1 е , ther every frequency 
Scores 1$ compatible with the Gaussian error model. 
*Ferguson has shown that, wi 


› With improbable excepti, i 
linear unless the tr probable exceptions, this re 


i istri i i be 
че score is normally distributed in the Ero Passion Нов 


up tested [7] 
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It must be admitted first of all that this model can never be strictly appropri- 
ate for describing an observed frequency distribution of number-correct or 
proportion-correct scores, simply because such scores represent a discrete 
and bounded variable whereas the scores produced under the Gaussian 
error model are necessarily continuous and unbounded (except, of course, 
in the degenerate case when the variance of the errors is zero; this case will 
be ruled out of further consideration). Avoiding a strict interpretation 


of the Gaussian error model, suppose we replace the histogram of the number- 


correct scores by a smooth frequency curve that approximates it, running 
from minus infinity to plus infinity. Will the model always be compatible 
with such a smooth frequency curve? It is intuitively clear that given some 
fixed value for the variance of the errors of measurement, there must be 
some limit to the sharpness of curvature in the frequency distribution of 
observed scores. Тһе presence of errors of measurement necessarily obscures 
in the observed-score distribution any sharp detail that might exist in the 
true-score distribution. There is therefore a limit to the sharpness of detail 
that can appear in the observed-score distribution under this model. Necessary 
and sufficient conditions for the frequency distribution of observed scores 
to be compatible with the Gaussian error model are given by Pollard [25] 
and by Standish [29]. In the present context, their conclusions are probably 
primarily of theoretical rather than practical interest. 
The Gaussian error model is completely represented by (9: 


м кә) = |. әуе - 9 d& 


where f(x) is the frequency distribution of observed scores, g(£) is the unknown 
frequency distribution of true scores, and N(z — 5 is the normal distribution 
for the variable e = х — £ having zero mean and known standard deviation, 
с. It is helpful to visualize this equation as referring to а scatterplot between 
true score on the abscissa and observed score on the ordinate. The quantity 


under the integral sign is the product of the frequency distribution of true 
; distribution of observed score for а 


Scores and the conditional frequency ion de 
given true score. This product is equal to the bivariate distribution between 
true score and observed score; for fixed values of z and & the quantity under 


the integral sign may be thought of as representing the frequency in any 
cell of the scatterplot. The integral sign represents a set of ееси тин The 
equation says simply that the frequency of occurrence of cy en ar score 
t is equal to the quantity obtained by summing all the cell frequencies in 
the corresponding row of the scatterplot. 
Equation (9) is an integral equation. The qu 
convolution of g(t) and N(é). The mathematical 
estimate g(£) when f(x) is known. More specifically, 
tion of the first kind. At the moment, it seems to me t 


antity (а) is known as the 
problem is typically to 
(9) is a Fredholm equa- 
hat methods for solving 
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such integral equations ([3], ch. 14; [4]; [31], sec. 3.15; and [5]) will be an 
important aid in making inferences about true scores. Trumpler and Weaver 
([32], chs. 1.4 and 1.5) give at least five different methods for solving equations 
such as (9). In addition, they go on to more advanced problems that are of 
great interest here, such as “Correction of a Bivariate Distribution for 
Observational Errors” and “Statistical Determination of the Functional 
Relationship between Variables.” 


Almost all methods for solving (9) start off by replacing the continuous 
variable £ by a discontinuous varia 


ble. For example, suppose that the true- 
score distribution g(£) can be approxima; 


ted by a discontinuous distribution 
denoted by G(£). Equation (9) becomes 
(10) Қа) = У GE)NG@n — Е), 


where т„ is any specified value of the observed score, 
of specified values for the true score. For fixed values о 
У(а, — &) is a tabled value of the normal curve or 
is à linear equation in the unknown frequencies, the 


and £, is one of a set 
f z and £ the function 
dinate. Equation (10) 


а апх simplicity of (10). If the number 
of equations is equal to the number of unknowns, then the solution merely 
involves computing the inverse of the matri 


rix whose elements are the values 
аз been com: 


seri 
ion wi 


represented by the С 


: $ may have а very irregular 
em is findin 


5 procedures that will deal 


ng variance of these estimators, 
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i 1 think you vill be interested in seeing how simple Gaffey’s method is. 
t will be convenient to describe the method in a practical application 
although the method itself is perfectly general. The data used were the 
observed scores of 388,071 students on а 50-item test. These data were chosen 
because of the large number of cases and because the score distribution looked 
approximately normal. Only .0002 of the cases made perfect or zero scores. 
The procedure used was to group the frequency distribution into 26 class 
intervals and then to compute the first, second, third, and fourth differences 
of the column of frequencies. Under the Gaussian-error model, any true-score 
frequency can then be estimated by starting with the corresponding observed- 
score frequency, subtracting from it one-fourth of the corresponding second 
difference, and adding one thirty-second of the corresponding fourth difference. 


I have gone into detail on this method in order to indicate what a very 


simple procedure may be found for estimating a true-score distribution 


directly in terms of the frequencies of observed scores. Unfortunately, the 
results obtained by the application of Gaffey’s method ,to the data described 
show that these data do not meet the assumptions of the method. The 
estimated true-score frequencies formed а bell-shaped distribution and 
appeared at first to be satisfactory, but the variance of the estimated distri- 
bution was found to be much too large in view of the known variance of the 
errors of measurement in the data. Dr. Gaffey was kind enough to correspond 
with me on this problem. It appears likely that the discrete and bounded 
character of a frequency distribution of number-correct scores cannot be 
reconciled with the assumptions underlying the Gi 


aussian error model within 
a sufficient tolerance to allow the application of Gaffey's methods. It is to 
be hoped that the binomial model, to be discussed next, will ау 


oid some of 
these difficulties. 

Before leaving the Gaussian error model, let me describe briefly a 
beautiful result obtained by Eddington and summarized by Trumpler and 
Weaver ([32], pp. 128-131). Eddington derives the regression of true score on 
observed score in the form 
(1) meme n 
re for a given observed score, 
т) is the first derivative 
This equation is based 
s other than the basic 


Where the term on the left is the mean true Sco: 
о? is the variance of the errors of measurement, and f 
of the frequency distribution of the observed scores. 
9n no approximations and involves no assumption: 
assumption of the Gaussian error model. 

How effective (11) will be for estimating true scores from number- 
correct observed scores remains #0 be determined. Numerical methods 
are certainly available for approxi st derivative from the first 
and higher differences of the observe bution. In any case, results 
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such as (11) give encouragement to efforts to find simple, direct methods for 
estimating true score from observed score without assuming a linear regression. 

Although it is clear that the Gaussian error model cannot fit number- 
correct score data exactly, it is of interest that data have been found for 
which the Gaussian-error model is rejected by appropriate tests of statistical 
significance. As already mentioned, the necessary significance tests were 
derived from the very general matched-forms model, which I discussed first 
of all. The significance tests were applied to the vocabulary scores in four 
widely different groups of one thousand examinees each. The Gaussian-error 


he four groups at the .01 significance 
[18]. These results do not mean that 


they do justify an attempt to find a more appropriate model for estimating 
true scores. 


(12) f) = |, «oi, a) ae, @ = 0,1, +. 


where ¢ is the proportion. 


(13) 


-correct true score 


$ = tn, 
and where B(t, x) is the binomial distribution 
(14) Bt, a) = ("ea — p=, 


This model is intended to be appropriate for dealing with number-correct 
scores, not for scores obtained by formula scoring or other methods. What- 
ever its faults, there seems to be 


ms t no question but that the binomial error 
model is a better approximation than is the Gaussi ш 


їп шапу зуауз а fairly simple mod 
For one thing, it is not diff he moments of the true- 

score distribution in terms of the moments of the observed-score distribution: 

(n — г)! 

(15) ut = p 1, (@=0, 1, +++ m). 

The left side of 


(15) represents the rt 
The quantity m 


s h moment of the true 
111 (£) designates the 


-score distribution. 
rth factorial momen 


t of the observed- 


FREDERIC M. LORD 297 


score distribution. A faetorial moment ([13], pp. 56-60) is simply a linear 
function of the regular moments, m; ; for example, 


(16) mla(z) = mi — бт; + limi — бт. 


It turns out that (15) is the same as the set of equations illustrated 
by (5), representing the rationally equivalent forms model. It is thus seen 
that the binomial error model is completely equivalent to the rationally 
equivalent forms model, insofar as the latter is concerned with predicting 
the shape of the frequency distribution of true scores from that of observed 
scores. The two models are not interchangeable, however, for purposes of 
inferring such things as the scatterplot between true and observed scores. 

For some time now, I have been trying out various numerical methods 
for using equation (12) to infer the frequency distribution of true scores from 
a given distribution of observed scores. One set of data may be of interest 
here. They are the scores of more than 2000 professional people on a 30-item 
figure matrix test. An interesting question arises because a full one-sixth 
of the examinees have scored at the chance level or below—a full 10 percent 
score below chance. Only 7 percent of the examinees get as many as half 
the items right after correction for guessing. The presence of so much guessing 
in the score distribution must tend to obscure the shape of the true distri- 
bution of competence in the group tested; so it will be of interest to examine 
the distribution of true scores. A still more interesting question is whether 
or not the scores below the chance level simply represent random deviations 
from a true score at the chance level, or whether some of the examinees 


actually have true scores that are below the chance level. I already have 
d question. There is no doubt but 


enough data clearly to answer the secon 

that a substantial number of examinees have true scores below the chance 
level. This result could occur in one of two ways, both of which seem to 
deserve further investigation. On the one hand, it may be that some examinees 
do not understand the directions to the test and systematically proceed in 
the wrong fashion. The second, and I think more likely possibility, is that 
many of the distractors for the test items are 50 ingeniously contrived as to 


be more attractive to some people than are the correct answers. 
ет peop r model to data of this kind where 


In order to apply a binomial erro r thi 
there are many omitted items, it is necessary for the statistician to supply 


а response for each item that each examinee has omitted. In the case of 
five-choice items, the result is the same as would have been obtained if the 


examinee had tossed a five-sided die to determine which response to make 
tions can be raised as to the legitimacy 


on the omitted items. While certain ques e rais 
of this procedure, it seems that it is not likely to give rise to true scores below 
the chance level unless these were already present in the em. inn 
Whenever the frequency distribution of true scores has been m 
from a given set of data by means of an integral equation such as (9) or (12), 
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it is always possible to use the integral equation to reconstruct the bivariate 
frequency distribution of true scores and observed Scores, since this bivariate 
frequency distribution is simply the integrand on the right side of the equa- 
ton. Given the bivariate frequency distribution between observed score 
and true score for any given set of data, it is а simple matter to obtain by the 
numerical methods the mean true score corresponding to each different 


observed score—in other words, the regression of true score on observed 
score. 


A simple recurrence relationship, 


apparent; i 
be derived from (12): Pparently as yet unpublished, can 


М... = ® MfG 4 1) 

(17) И: z ntc z f(x) (1 Е: М... ). 
In words, the mean true score for a; 
from the frequency of occurrence 


ib ТУО T dealing with this in- 
18 wholly reasonable to require that 
еп 2 and the estimated value of M... be kn en 


tion. This requirement 


5 to the values that can be assumed by 


m 
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M... Figure 1 shows two different curves obtained for M;.. by choosing 
the values of ЛГ, 1, first as large as possible and then as small as possible 
consistent with monotonicity in the region « = 10, 11, 12. The figure suggests 
that for these data the average true score can be determined within a fairly 
narrow range for any given observed score. 


Ух 
10 
5t 
F 
0 а + тоа ERR екса TD 
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Ficure 1 
Two estimates of the regression of true score, ton observed score, 2 (М = 4,000) 
It should be explained that Figure 1 is derived from the observed-score 
distribution of 4,000 examinees on à 25-item vocabulary test. The observed- 
Score distribution was smoothed before carrying out the computation, using 
а five-point formula given by Cureton [6]. Otherwise even with 4,000 ex- 
aminees, the regression of true score on observed score would have been 
excessively irregular, due to chance fluctuations in the adjacent frequencies 
of the observed-score distribution. 
The jagged line in Figure 2 is 

on observed score for а 50-item 4 
388,071 examinees. The observed-score 
Obtaining Figure 2. The figure is presente! 


the regression of true score 
uantitative test. In this case, there were 
distribution was not smoothed in 
d to show the excellent agreement 


an estimate of 
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05: 


30 40 50 X 


Ficurn 2 
Regression of true score on observed score estimated from К-В, (21 


) (straight line) and 
from binomial error model (№ = 388,071) 


In the case of Figure 2, there 


appears to be і 
since the regression is v 


) ttle need for using (17) 
ery close to linear and can 


be well represented by 
guessed in advance from 


etermine the values of М;.. so 
ession line in some least squares 
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sense. For other methods related to this problem, see Robbins [27] and 


Steinhaus [30]. 
Much more remains to be done in working out convenient numerical 


methods for dealing with the relatively simple problems already discussed. 
for which test theory should 


There are also certain more advanced problems, 

be able to provide an answer. It is hoped that the present methods may lead 

to solutions for some of the following, for example. 
1. Given the observed-score distribution for a single group of examinees 

on each of two tests that are known to measure the same psychological 

dimension, but that are not otherwise parallel, estimate the possibly curvi- 

linear functional relation between their true scores, and deduce from this 


the seatterplot of their observed scores. 

‚ 2. Given, further, the observed-score distribution of one of these two tests 
in а second and somewhat different group of examinees, estimate the observed- 
score distribution of the other test in the second group of examinees. Also 


estimate the scatterplot for the second group of examinees. 

3. Devise a method for determining whether two tests do or do not 
measure the same psychological dimension, ie., à method for determining 
Whether the true scores on two tests have à perfect curvilinear relationship 


ог not. 

In closing, I would say that all five of the true-score models outlined 
yield good results when applied appropriately. All five will probably con- 
tinue to be of active interest. Empirical studies comparing the results obtained 
under different models would help to clarify their various advantages and 


disadvantages. 
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the understanding of human behavior in un- 


A theoretical approach to 
is suggested, ап approac 


certain outcome situations is ) h 
utility theory, decision-making theory, and statistical association theory. 


Experimental evidence вир] orting this approach as opposed to alternative 
approaches is summarized. Three different formalizations are prese! 


а variety of experimental tests is suggested. 


The purposes of this paper are: 
I. to discuss a controversy wh 
approaches pertaining to the behavior 0 


involving choices under uncertainty; 
IL. to suggest a different approach, set in à decision-making and utility 


theory framework, to account for the choice behavior to which both the 


other theories have reference; А 1 
ІП. to present three mathematical models—each involving concepts 
from decision-making and utility theory—which make predictions about 


human behavior in the choice situation under discussion; 
IV. to present and discuss some experimental evidence which supports 


the general features of these models. 


ich has arisen between two theoretical 
f human beings in a situation 


The Controversy Between Two | 
Theoretical Approaches to Choice Behavior 


The predictions which people make when placed in а two-choice un- 
certain outcome situation ha ved considerable attention in recent 


years [2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17, 18, 20, 21]. In the classical ex- 
perimental situation, as first used by Humphreys [14], the subject E asked 
to predict whether a light will or will not appear after à signal stimulus. 


Following the signal, either of two mutually exclusive alternative events 
the Center for Advanced Study in 


*This paper was written while the author was at the e 1 
Э tly from discussions with many colleagues 
benefited Мо II emerged from work with 


the Behavioral Sciences, and it has о аа 
f resent state partly as & result of 


there. In particular, some о 

Robert PAb ‘und Model III was developed to its p! 

е от Shannon, n e E Ap Deperpened during many 
at Stanford University). The treatment of all three models was sharpene g many 
discussions with Robert M. Solow. 


ve recei 
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can occur: the light can appear or the bulb can remain unlit. The two events 
occur with fixed but unequal probabilities, say m, and т» , ina random sequence 
for à number of trials. In recent years, two lights have often been used, one 
of which appears with probability т and the other with probability 1 — т 
on each trial. In some cases, events other than lights have been used. In 
any case, the events are independent of the subject’s behavio 
of what he does, one of the tw: 


rectness or incorrectness of ea 

There are two theoretical 
about an individual’s behavio 
statistical learning model [5], 


; Tand 1 — т. The same prediction 


To contrast the predictions from the two models, consider, for example, 
a two-choice uncertain outcome situation in whi : A of trials the 
two events, F, and E, , occur with probabiliti 
respectively. Estes' model asserts that a p 
tions which will tend towar 


E.-pr- — ра = т), 
7575) + 25025) = боз. 


І 


On the other hand, if 
dicting E, on every tr 
portion of correct рге 


the subject were to adopt the « 


ial (p — 1.0), he would h 
dietions, for 


pure strategy” of pre- 
ауе .75 as his expected pro- 


1.0(.75) + 0(.25) = 75. 


One straightforward predietion from the Same-theoretio model then, would 
3 ? 


D^ 


== 


ИЯ ө Ҥ ---ө ө ө 
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€ сау, а peiton would behave in the second way. Evidence giving 
и = | 18 prediction may be found in [10, 11, 17, 18, 21]. 

iin have ied үт casar ei sketched above, some behavioral scientists 
people who eei р by game-theoretic principles have asserted that 
ae Е te heir response ratios to the probabilities of the events 
proportion "e 1006 y, in that they are failing to maximize their expected 
= заема ч correct predictions, a goal they could accomplish by predicting 
ies a event on every trial. The empirical fact is that in this 
Done тай е people do, after many trials, stabilize at matching their re- 
tional is't Eon о the probabilities of E, and F, . To assert that this is irra- 
diy В edi А опа highly restrictive meaning of that term. It was pointed 
of about TS Ш іп е first half of the eighteenth century that any theory 
e pido : ehavior which does not incorporate the concept of the utility 
ipse а value of the outcomes rather than their objective value will lead 
5,271) хез of the kind under discussion. Аз Simon has reminded us ([20], 
pas суй must bear in mind the distinction between objective rationality 
incon p y as viewed by the experimenter) and subjective rationality (be- 
ilis c hat is rational, given the perceptual and evaluational premises of 
e ge ject). The purpose of the following section is to suggest а resolution 

apparent paradox which has been described, à resolution which in- 


Corporates the utility theory approach. 
A Decision-Making Approach 


a z rom decision-making theory [3, 4], & hypothesis of maximization of 
m utility may be drawn which will account for both sorts of prediction 
; aviors—matching response ratios and maximizing the expected frequency 
correct predictions. According to this approach, whether а person will 

ond toward one or the other prediction strategies depends on certain con- 
ditions related to the reinforcement inherent in the situation. Where utility 
is understood to refer to the subjective value of an outcome, the general 


hypothesis is that a person will behave as if he were attempting to maximize 
he components of the total utility vary 


expected utility in any instance. Т 

m magnitude from situation to situation and thus a person's strategy to 

maximize expected utility varies. 
It is reasonable to suppose t 

the only payoff attached to the 

predietion confirmed by the event or the dissati 


diction disconfirmed by it, making a correct r^ та 
has greater utility for the person {һап making a correct prediction of the 


more frequent event. The person derives satisfaction from playing à game 
With the machine, trying to outwit it. Moreover, there is the matter of mo- 
notony, both kinesthetic and cognitive: predicting the more frequent event 
on all trials would engender the monotony of pressing the same button (a 


person is in a situation in which 
в the satisfaction of having his 
sfaction of having his pre- 
diction of the rarer event 


hat when а 
outcomes 1 
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common method of stating predictions) on trial after trial for hundreds of 
trials, in addition to the monotony of the same cognitive response (e.g., left 
light) on trial after trial. Under such cireumstances, а subject may maxi- 
mize expected utility by matching his response ratio to the actual prob- 
abilities of occurrence of the two events. For him, the cost of an incorrect 
prediction is very low, whereas the gain from this strategy in terms of other 
utilities, such as the utility of gambling and the utility of variability, may 
be relatively high. By choosing a mixed strategy (i. e., splitting his predic- 
tions in some proportion), such a subject may maximize his own total satis- 
faction. If such an account is correct, then a decision-making model and 
the Estes model could yield the same predictions concerning the stable state 
(asymptotic) behavior of a person in a two-choice no payoff situation. 
However, if the utility attached to correct predictions is increased by 
a change in the conditions of the game, i.e., if there is potential satisfaction 
beyond that of having an event confirm one’s prediction, or if the cost at- 
tached to an incorrect prediction is increased, then the prediction yielded 
by the hypothesis of maximization of expected utility diverges from the 
prediction yielded by the Estes model. That is, if the decision-making theory 
approach proposed here is correct, the introduction of systematic variation 
in the reinforcements (and thus utilities) attached to correct or incorrect 
predictions should be followed by system: 
As the utility of a correct prediction is 
utility) of an incorrect prediction is i 
the more frequent event should tend to 


| ; ) it is necessary to have 
more precise formulations of the decision-making theory approach than the 


Alternative Utility Models о] Choice Behavior 
: A e three models which are to be presented, Model I is the simplest. 
entra ili 


subjects, there is little, if any, 
since the utility of gambling 
others. 


of averages for groups of 
m the utility of gambling, 
€ people and negative for 


systematic effect fro 
is positive for som, 
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Model II is richer than Model I in that it includes the concepts of the 
first model—although slightly different mathematical properties are ascribed 
to them—and it also deals with the utility of gambling. 

Model III contrasts with the first two in that it incorporates the Shannon 
information measure, and also in that it seems to lead to somewhat different 


predictions than are yielded by either Models I or II. 


Model I 
Let т = the probability of occurrence of the more frequent event, 
p = the proportion of times the subject chooses the more frequent 


event, ae 
a = the marginal utility of a correct prediction, and 


b = the marginal utility of varying one’s responses. | 
If the expectation that а subject’s prediction will be correct, E; , is 


Е. = [рт + (1 - pa - 7] 
= @-э + per - 11, 
then the expected utility of а correct prediction, Uz ws 


v= of, = aft = + pr — DT 
sponses = f(p), such that U, is sym- 
p(l — P), (which, though arbitrarily 


I U, = the utility of varying one’s re 
metrical and maximal for p = +5, €£« 


chosen, has considerable intuitive appeal), then 
U, = bp — 0: 
The total expected utility of а particular strategy р 13 
оф = 0. + U. 
= [1 —1) + р(2т — 1] + а — D. 

The strategy p which maximizes expected utility is at 

ай) _ о 

ар 

апа is 

ат — 1) u 

p= +2 

If = a/b then 
0 poor 9 ++ 


d ошу when а = b and thus а = 
ity of a correct prediction equals 


It may be seen from (1) that when ап! 
Model I makes the same 


1, р = т. That is, when the marginal util 
the marginal utility of varying one’s responses, 
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prediction regarding asymptotes (stable state behavior) as the Estes model. 
This situation seems to hold when the only reward in the situation is know- 
ing whether one's prediction is correct or not. However, if the reinforce- 
ments inherent in the situation are increased, say by adding monetary re- 
wards and/or costs to the outcomes, and thus the utility of a correct predic- 
tion and/or the negative utility of an incorrect prediction increases, then 
а > banda > 1. For such conditions, the predictions regarding asymptotes 
yielded by Model I diverge from the predictions yielded by the Estes model. 

The predictions Model I yields concerning a subject’s choice and strategy 
behavior may be generally stated as follows. For any т, 


(i) p = 1 when a > 


2r — 1, 
(ii) 1 > p > т when <<, 
(Ш) p = т when a = 1, 
(iv) p < т when a < 1. 


The third prediction, that p = т, is the prediction yielded by the Estes 
model, and is a special ease of Model I. The first, second, and fourth predic- 
tions are ones which it would seem that the Estes model is not prepared to 
make. 

In Model I, & may be estimated from data from (1), 


_ P= (1/9) 
oe 22 (1/2)` 


In the final section of this paper, experimental results are reported 
which give some support to Model I. It will be seen that when Ss are run 
in the Estes situation under three different reinforcement, conditions, the 
values of @ which may be estimated from the data are: i 


No Payoff condition, & = 1.00; 
Reward condition, & = 1.44; 
Risk condition, @ = 1.80. 


A relatively simple test of Model I is being conducted by the author at 
present, in which the above values of & are utilized under different values of 
т. Preliminary results strongly support Model I. 

A more crucial test of Model I would entail the experimental manipu- 
lation of а by variation of b, the marginal utility of varying one’s responses. 
In the Estes No Payoff condition, with an increase in b, one would predict 
р < т. With a decrease in b, one would predict p > т. S. Үү. Becker, in à 
personal communication, has described data he has collected at Stanford 
University confirming these predictions. 


- 


LESER eee 47) ЕЕ Е 


View 


Ej — 
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Model П 
Let т = the probability of occurrence of the more frequent event, as in 


Model I, 

p = the proportion of times the subject chooses the more frequent 
event, as in Model I, 

а = the marginal utility of a correct prediction when and only 
when the subject chooses the more frequent event, 

b = the marginal utility of a correct prediction when and only 
when the subject chooses the less frequent event, and 

c = the marginal utility of varying one’s responses. 

The expected utility of a correct prediction, E.(U,), is 


EAU,) = арт + 51 — Ра — т), 
and the utility of varying one's responses, U, , is f(p), as in Model I, and is 
С, = ep(1 — р). 
The total expected utility of a particular strategy p is 
Гр) = арт + b(1 — p) — т) + а — р). 
The strategy p which maximizes U(p) is at 


dp И 
апа іѕ 
_ ar — b1—57)-c 
pe 2c 
Па = a/c and if B — b/c then 
(2) р = = Bt se t 9. 


It may be seen from (2) that when a = b = c, and thus a = 8 = 1, Model 
II yields, as a special case, the Estes prediction that p = т. It will be shown 
later that the case when о = 8 = 1 is not the only case for which p = т 
according to Model II. | 

Model II is richer than Model I in several respects. One is that it makes 
Dossible experimental studies involving differential payoffs for the more 
frequent and less frequent events. In addition Model I may lead to the 
measurement of an individual's specific utility of gambling. The predictions 
Model II yields concerning a subject's choice and strategy behavior may 
be generally stated as follows. For any 7: 

(i) p = 1 when 


апа/ог 
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o mem 1 (А) 
т 


semi, e | 
(i) 1 > p > «when | 
( т)8 +1 (1 т8 + (2x — 1) 
T TRE M | 
and/or 
ерау. | 
шы i 1-х 
(iii) р = + when 
а = Ч- 0) + (2r — 1) 
= TPT wm — 1) 


z (A.) " 


апа/ог 


porn (B.) 
Gv) р < + when 


a < (L— 98 + (2s — 1) 


and/or 


8>14- От ~ 1) 
— 

The first, second, and fourth 
т, or less than т—аге predicti 
make. The third prediction—t; 
and occurs here as a special с: 
note that the second set 
inequalities (A,) > (А,) 
In Model II, there 


are two parameters, œ an 
mated before the value o 


е d В, which must be esti- 
f p can be predicted: 


(2) р = RHO + ret 9). 


To test Model II, two successive studies Would be needed, the first to obtain 
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estimates of the parameters. Subjects could be run under two т” and one 
level of reinforcement, №, . Data would be required, that is, from a study 


of the following design. 


With т, and т» set, and with pi and p, observed, according to Model II, 
(1 — В + m(a + 8) 
2 ? 


(1 — 8) + mle + B), 
2 


р = 


Solving these two equations simultaneously, estimates of 8 and a are as 


follows: 
8 - Np сере) ЕЕЗ, 
a= 2(m = рз) ety 8. 
т: — Te 


With д and 4 computed, а second study could then be conducted using а т 
different from those above, вау 7s - Under the same reinforcement, В, ,as 


was used above, a test of Model II would be a test of the prediction 


(1 — 8 4 (e +В. 

puc 2 

ween Model I and Model II is that Model 

II contains separate expressions for the utility of correctly predicting the more 

frequent event and the utility of correctly predicting the less frequent event. 

Thus, whereas both models predict that p will depend on the reinforcement 

hinging on correct predictions, only Model II yields а prediction for the 

situation in which the reinforcement contingent on correctly predicting the 
ontingent on correctly 


more frequent event differs from the reinforcement с 
predicting the less frequent event. According to this model, by systematically 


varying these two types of reinforcement it should be possible to induce 


variations in p from zero to one! з \ t 
With such an experimental set-up it should be possible to obtain а 
tility of gambling. To obtain such a measure, 
bject/s utility of money, and this could 
d by Davidson, Siegel, and Suppes [3]. 
large а change in money is 
from some stable value to 


Ап important difference bet: 


measure of a subjects specific u 
it would be necessary to know the su 
be determined by the method develope: 
The experimenter could then determine how 

necessary to change the subject’s strategy, P, 
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unity. This result, under the proper experimental conditions, could yield а 
measure, in monetary units, of the subject’s specific utility of gambling. 


Model ПТ 
The third model makes a potentially interesting connection between two 
theoretical approaches: utility theory and information theory. 


Let т = the probability of occurrence of the more frequent event, as in 
Models I and II, 


p = the proportion of times the subject chooses the more frequent 
event, as in Models I and II, 


a = the marginal utility of a correct prediction when there is a pay- 


off (say monetary) in addition to the satisfaction of knowing 
a prediction has been confirmed, and 


b = the marginal utility of "reflecting" or “reproducing,” in some 
manner (stochastic), the information in the event system. 
The expectation of a correct prediction, Æ, , is 
Е. = [pr + (1 — Ра — т)] = [а — т) + plan — 1)]. 
Then the expected utility of a correct prediction with payoffs over and above 
the satisfaction of knowing a prediction has been confirmed, E.(U,), is 
EÁU,) = а[1 — 7) + рт — 1)]. 


The logarithm of the expectation that the subject w. 


| ill reflect the infor- 
mation, or stochastic structure, of the event system, for 


n trials, is 


n T" = (1—--)n 
tog (^) A =y . 


Utilizing Stirling's formula and expanding, the logarithm of the expectation 
for any particular trial (е. n = 1) is 


[т log p + (1 — т) log (1 — »]-— 


[т log т + (1 — я) log (1 — т)]. 
"Thus the expected utility involved is 


U: = bile log p + (1 — я) log (1 — P?) — [r log r + (1 а) log (1 — 3)]), 

and the total expected utility of a particular strat 

ОФ) = Un + О, 

all — т) + р(2т — 1)] + br logp + (1 — 
— bir loge + (1 — т) log (1 — 7)]. 


The strategy р which maximizes expected utility can be fou 


egy p is 


1 


7) log (= 2)] 


nd by Setting 
dU(p) 


dp — 0; 
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that is, 

а(2т — Пр’ + [b — а(2т — D]p — br = 0. 
By dividing by b and letting а = a/b, 
(За) alr — 1)р* + [1 — ат — D]p — т = 0. 


From (3a) № may be seen that under the conditions of the conventional 
statistical learning experiment, conditions in which there is no payoff other 
than the satisfaction of knowing the correctness or incorrectness of a pre- 
diction, а = 0 and thus р = т. That is, the Estes prediction is yielded for 
such conditions. However, if reinforeement is increased, then, setting Х = 
2r — 1, 


Е —(1— a) + wv(1- ad) + dadz 
дол 


(3b) p 


For three commonly used values of т the predictions that Model III 
yields concerning a subject's choice and strategy behavior are shown in 
Figure 1. It is interesting to notice that, in contrast to Models I and II, 
Model III does not predict that p = 1 for any finite value of а; rather, in 
Model III, p approaches 1 asymptotically as a increases. 


Relevant Experimental Evidence 


Experimental data are available to support the general approach to the 
prediction of asymptotes shared by these models as opposed to the general 
features of statistical association theory. That is, there are data supporting 
the notion that an increase in the reinforcement contingent upon a correct 
prediction, and thus the utility of a correct prediction, should result in a 
subject’s р being higher than т, and that under high reinforcement p should 
approach unity. These data are presented below. They are of interest here 
for the additional reason that the value of а can be estimated from them and 
this estimate may be used in future studies to predict р (from Model D. 

A study was conducted by Siegel and Goldstein [19] in which the ex- 
perimental situation already described was used and in which subjects were 
observed under three conditions of reinforcement: No Payoff, Reward, and 
Risk. Two lights were illuminated according to a random series, with one 
illuminating 75 percent of the trials (т = 75) and the other 25 percent of 
the trials (1 — т = .25). The subjects were 36 male students. They were 
randomly assigned to three sets of equal size, the assignment determining 
the condition under which the subject would be run. The three conditions 
were as follows. | 

No Payoff. Under the No Payoff condition, the reinforcement for each 
prediction consisted simply of seeing the outcome—seeing whether the right 
or left bulb lit, and thereby determining whether one's predietion was con- 
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Ficure 1 
Predicted values of p from (3b) with а = 0 to 10 for three levels of x, 


firmed or disconfirmed. The No Payoff condition has been the “classical” 
situation for studies of human behavior in two-choice situations, 

Reward, Under the Reward condition, the r 
diction consisted of seeing the outcome and re 
correct prediction. The reward was given at the 
which S's prediction was confirmed. 


Risk. Under the Risk condition, the reinforcement for each prediction 
consisted of seeing the outcome, receiving five cents if the prediction was 
confirmed, and losing five cents if it was disconfirmed. At the conclusion of 
every trial, S either received or forfeited five cents, depending on whether 
his prediction had been correct or incorrect. 

Every subject, regardless of condition, was given 75 се 
of the study and told that whatever money he held at the ane 
be his to keep. 


The purpose of the experiment was to test a hypothesis drawn from the 


einforcement for each pre- 
ceiving five cents for each 
conclusion of each trial in 
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decision-making approach introduced earlier, a hypothesis which reflects the 
general approach of the three models which have been presented. The hy- 
pothesis was that the asymptotic probability of a person’s predicting the oc- 
currence of the more frequent event in a two-choice uncertain outcome situation 
is a function of the level of reinforcement present in the situation, such that the 
probability of predicting the more frequent event will tend toward unity as the 
rewards (positive utility) and costs (negative utility) of correct and incorrect 
predictions are increased. 

That is, using Estes’ notation [ef. 6], where ф,( =) is the mean asymp- 
totic probability of predicting E, for a group of like individuals, the hypoth- 
esis was that 

ji (=) under Risk > ji(9) under Reward > ф() under No Рауой. 

The data confirmed the prediction at p < -001 [16]. Ss under the Risk 
condition predicted the more frequent light oftener than Ss under the Reward 
condition, and these in turn predicted the more frequent light oftener than 
Ss under the No Payoff condition. Under the No Payoff condition, the mean 
proportion of times that the more frequent light was predieted during the 
final 20 trials of the first 100-trial series was 70. Under Reward, the mean 
was .77, and under Risk it was 93. 

For those 12 Ss (four randomly selected from each group) who were 
maintained under the same reinforcement conditions for 200 additional 
trials, the mean proportion of times that the more frequent light was pre- 
dicted during the final 20 trials of the final 100-trial series was .75 under the 
No Payoff condition, .86 under the Reward condition, and .95 under the 
Risk condition. There was no overlap among the scores of these Ss under 
the three conditions at the end of 300 trials. 
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ТНЕ СОМСЕРТ ОЕ CONFIGURATION OF INTERPERSONAL 
RELATIONS IN А GROUP AS À 
TIME-DEPENDENT STOCHASTIC PROCESS 


Leo Karz anp Cuartes Н. Procror 


MICHIGAN STATE UNIV ERSITY 


А To explain changes in the sociometric configuration of a group through 
time, a problem arises of the extent to which such changes may be viewed as 
the aggregation of part-processes occurring at the level of two-person choice 


structures. A possible model is a Markov chain in which three possible states 
are mutual choice, one-way choice, and indifference, one realization for each 


pair of choosing individuals in the group. Choice data for an eighth-grade 
classroom are fitted to this model and are used to answer questions of con- 
stancy of transition probabilities, order of the chain, and sex differences. 


While the full complexity of the notion of the configuration of inter- 
personal choices as representative of a social group has been frequently 
discussed in the literature, most authors (including the present ones) have 
quickly abandoned attempts to deal with the process a$ à whole and have 
instead concentrated upon particular facets of the process. Although practi- 
cally everyone recognizes that the process is one which evolves and modifies 
in time, almost all past investigations have been couched in terms of single 
points in time. The literature is replete with examples of studies leading to 
the description of the configuration in а static way; there exist only à very 
few examples of attempts to deal with as many as two observations of the 
configuration, separated in time. 

In this paper a view of the configuration as à time-dependent process 
is specified and certain simplified stochastic models are developed, retaining 
all of the essential features of the process. Finally, an actual example con- 
sisting of а series of observations on the configuration for а single group is 
considered, and tests are applied to see whether the models describe the 
process reasonably well. 


Configuration of Г nterpersonal Relationships as a Time Process 


The standard sociometrie definition represents a group in terms of the 
aggregate of all of the interpersonal relations between pairs of individuals 
constituting the group. Such an aggregate of relations is called the configu- 
ration for the group; we recognize that such а description is valid only at a 
specific point in time. In this sense the configuration of the group is a func- 
tion of time, and changes with time. Ав time proceeds the configuration 
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changes kaleidoscopically with some of the interpersonal relations remaining 


unchanged, while others change abruptly from one quality of the relation- 
ship to another. 

А particular relation between a designated 
point in time, is considered to be in one of a discr 
the various possible qualities of this interper: 
proceeds, this state continues in force until some 
changes abruptly to some other State, continui 
the next change, which may be a return to th 
to а new state. 

Alternatively, 


pair of individuals, at any 
ete set of states representing ' 
sonal relationship. As time 
subsequent point at which it 
ng in the second state until 
е previous state ога passage 


one may wish to look at the entire configuration con- 
sisting of the aggregation of all of the pairwise relations, and consider transi- 
tions among the possible states of the configuration, However, if there are 
N individuals in the group and each relation may be in any of k states, the 


number of conceivable states of the configuration is к 2) а fantastically 
large number. With № = 10 and k = 3, this number is 3 X 10?! 


The Two-Person Relation 


ion which 
bservation, 
à i ges in two 
Steps, perhaps even returning to the i . Thi Y not be at all 
Serious since these states are of уе 1 


double 
with respect to 


etween the rela. 
individuals, The 


tions “on-off” and “off-on” 
differentiation is difficult to ess there is ас 
among the individuals in the group in terms of status ine 
2 еш : Б оше, е{с. 
instances it is essentia] to know whether а one-wa; i oes pu Шаю 
lower to the higher, or the reverse. If, for the 
this identification is not desirei 
two-person relation, the state i i 

2 choi 
choice, and the state of non-choice į istos 1, 


| 
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are designated by the numbers 2, 1, 0, respectively, corresponding to the 
number of positive choices expressed. 

Although it seems almost preposterous to hold the position strongly, 
logic demands initial consideration of a simple model in which the relations 
at different periods of time are independent. If this is the case then the prob- 
abilities of the relations 0, 1, and 2 are independent of the actual relations 
observed in earlier periods. When sociometrie measurements at two points 
in time are available for the same group it is possible to test the hypothesis 
of independence by examining the 3 X 3 array of numbers of pairs falling 
into the various combinations of relation before by relation afterward. If 
the numbers observed are sufficiently large, a simple chi-square test with 
four degrees of freedom is appropriate. Almost certainly, this test will yield 
rejection of the hypothesis of independence as а result of persistence of 
large entries in the principal diagonal of the array. 

If the process is not random and independent in time, one might next 
investigate the nature of the dependence of the state of the process at one 
point in time upon states at other points in time. Although it is possible 
that a case might be made for a more general mode of dependence, experience 
in other investigations would indicate that the next most likely model to be 
tried is some sort of Markov process, with the nature of dependence on the 
past so far unspecified. A rather complete exposition of the theory of dis- 
crete Markov processes may be found in Feller ({2], ch. ХУ and XVI). At 
this point it is possible to propose & variety of different tests which vary with 
the specific nature of the Markov process under consideration. It is not 
possible to outline all of the tests which might be made; instead, a particu- 
lar collection of data will be examined and how one may determine the de- 
pendence of the process on the past will be indicated by example. 


Empirical Examination of the Markov Properties 


Data consist of the choices made by an eighth-grade class at four time 
points in the school year reported by Taba as part of a project in inter- 
group education [3].* Twenty-five students, 16 girls and 9 boys, were asked, 
“With whom would you like to sit?” This sociometric test was administered 
in September and November and in January and May of the following year. 
All pupils made three choices each except in November when two gave only 
two choices each. A complete description of the research and an able analysis 
of the static group structure is available ((3], рр. 45-75). 

Frequencies of the various transitions in the 300 two-person relation- 
ships between pairs of time points, which appear in Table 1, allow one to 
test whether there is independence in the sense of uniformity of distribution 


*The data were used in this study with the kind permission of Dr. Taba. 
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TABLE | 
Distributions of Types of Transitions in Two-Person Choice Structures, 1 
Test Population of 25 Eighth Graders | 
| 
Time Periods 
Type of Transition 
Sept. to Ном. to Sept. to Jan. to Nov. to Sept. to 
Before After Nov. Jan. Jan. May Hay May 
mutual 7 7 6 6 5 5 
Mutual to l-way 5 o 3 7 5 3 
indiff. 3 6 6 1 3 7 
mutual 5 7 6 6 3 3 
same I-way 17 18 1% 15 15 9 
l-way to opposite l- . 
мау 2 3 1 1 5 4 
indi ff. 21 18 2% 25 23 29 
mutual 1 0 2 % 8 8 
Indifference to l-way 22 26 29 20 18 27 
indiff. 217 215 209 215 215 205 
Sums 300 300 300 300 300 300 


between states of these relations at differ 


ent times. Tables su 
ing were examined (in each case, with 4 


ch as the follow- 
degrees of freedom) 


State of State in November 
relation in 
September 2 1 0 Sums 
Aer | ====— 
Mutual 2 7 5 3 15 
l-way 1 5 19 21 45 = 2 
IndifferenceeO | 1 — 39 giz 7 т=н 
Sums 13 46 241 300 
State of State in May 
relation in 
November 2 1 0 Sums 
Mutual 2 5 5 3 13 
l-way 1 3 20 23 46 А = 
Indifferenee0 | 8 1% "NE ТА 


Sums 16 


43 241 300 


Without” doubt, the later states of two. 
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? To make the time gap of the transition 


November to January transition? 
equal, the transition from September to November is compared with that 


from November to January and the September to January transition is 
compared with that from January to May. The test consists of computing 
a chi-square value, as in à test of association, for the various 2 X 3 and 


2 x 4 tables ([1], р. 98). An example is the following. 


Transition 
Starting from a Sept. Nov. 

Mutual and to to 

ending at... Nov. Jan. Sums 

Mutual 7 7 14 
l-way 5 0 5 х = 5.87 
Indifference 3 6 9 

Sums 15 13 28 


The larger the chi-square value, the more untenable becomes the assump- 


tion of constant transition probabilities. 
For the September to November and November to January comparison 


the chi-square values are: starting from mutual, 5.87; from one-way, .80; 
from indifference, 1.36. The comparison of the September to January with 
the January to May transition yields: starting from mutual, 5.13; from one- 
way, .00; from indifference, 2.40. Although there is a suggestion that the 
probabilities of reaching states from a mutual relation differ from one time 
to another, the chi-square values do not attain significance at the .05 level. 
Thus, we will proceed as though they were constant through time. 


TABLE 2 


Second Order Transitions іп Two-Person Choice Structures 
from September-November to January 


Structure in Structure in January 


mutual 4 0 3 7 

Mutual: 1-мау 2 3 0 5 
indifference 0 o 3 3 

1 я mutual 2 0 3 5 
-мау: 1-мау 4 9 6 19 
indifference 0 6 E 21 

tual 0 1 

indi FFarence: [eee 1 5 2 22 
indifference o 20 197 217 

Sums 14 47 239 300 
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We now ask if the Markov chain is of order one or order two. Гог 
this test the distinction between same and different one-way was ipsis e 
that there would be only nine transitions between the three states before ш 
after, а more conventional chain. The data used to make the test are foun 
in Tables 2 and 3. The present test likewise consists of making various chi- 


TABLE 3 


Second Order Transitions in Two-Person Choice Structures 
from September-January to Мау 


Structure in Structure in Hay 


September January Mutual 1-нау Indifference Sums 

mutual 4 | 1 ч 
Mutual: l-way 1 

Б indifference 0 1 5 6 
mutual 2 1 i Е 
Т-мау : 1-мау 9 5 
К indifference 1 4 19 2% 
mutual 0 2 0 2 
Indifference: l-way 5 10 14 29 
indifference 3 15 191 209 
Suns 16 43 241 300 


Square tests of association—in this case 


for 3 X 3 tables. For example, from 
Table 2 consider the following tabulatio 


n. 


Structure in Structure in J; anuary 


Sept. Nov. Mutual l-way Indifference Sums 
Mutual Mutual 4 0 3 М! 
1-уау Mutual 2 0 3 5 
Indifference Mutual 1 0 0 1 

-- 
Sums 7 0 6 13 


x = 1.26 (2 degrees of freedom because of 0’s) 


A large value of chi- 
of the struetures in September has influence 
new structures in J; anuary. That is, larger chi 
or higher order chain, The observed chi-sq 
"Table 4. 


Square would indicate that а difference in the state 


over the movement to 
“Square values indicate a Second 
uare values are Summarized in 


Obviously the chain of four-month 
does not show second-or 
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TABLE 4 
Values of Chi-Square for Tests of Order of Chain 


September-January to 
May Transition 


———— 


Septembe r-November to 
January Transition 


жж 


Structure at Intermediate Degrees of Degrees of 
Time Point Chi-Square Freedon Chi-Square Freedom 
Mutual 1.26 2 5.78 4 
l-way 8.02 4 4,26 4 
Indifference 7.82% 2 4.27 4 
17.25% 8 14.31 12 


Sums 


Significant at 5% level. The value 7.82 when corrected for continuity becomes 6.83, which is still 
significant at the 5% level. This correction consists of computing the next smallest value of chi- 
squared, in this case 5,83, for the same set of marginal totals and using as "corrected" value the 


mid-point of the two, or 6.83. 


other states in January. The following tabulation shows the direction of 
this influence (numbers in brackets are theoretical or expected frequencies). 


Structure in Structure in January 
September November Mutual l-way Indifference Sums 
Mutual Indifference 0 0 10.32] 3 [2.67] 3 
1-уау Indifference 0 6 [2.26] 15 [18.78] 21 
Indifference Indifference 0 20 [23.41] 197 [193.59] 217 


Sums 0 26 215 241 
ndifference does not eliminate the 


Apparently, the passage through i 
A similar but less marked tendency 


tendency to return to the previous state. 

can be noted in the chain of four-month gaps. Possibly the investigation of 
more closely spaced time points would reveal a considerable number of 
returns to a previous state even upon passage through a mutual or a one- 
way structure. 

Having decided that the chain of fou 
that the transition probabilities are stable, the data in Table 1 are used to 
estimate these probabilities. From а mutual structure the probabilities of 
becoming in four months a mutual, a one-way, or an indifference are .414, 
345, and .241, respectively; from а one-way to a mutual, same one-way, 
different one-way, or indifference are .130, .315, .022, and .533, while from 
an indifference to a mutual, one-way, ог indifference are .013, .102, and .885, 
respectively. The matrix of transition probabilities is constructed by com- 
bining the ‘о same” and “о different one-way” transitions. 


r-month gaps is of order one and 
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To 
сазы был А ы АНИ 
From 2 1 0 
о ыы >) 
Mutual 2 ‚414 .345 .241 
l-way 1 .130 .387 .533 
Indifference 0 . 013 .102 .885 


Since all states are accessible from each state, the Markov chain is 
irreducible and possesses a unique stationary distribution with limiting prob- 
abilities for mutual, one-way, and indifference of .051, .149, and .800, re- 
spectively. Among the 300 two-person structures in à 25-person group, one 
would expect to find as “stable” values 15 mutuals, 45 one-ways, and 240 
indifferences. It is thus not surprising that the numbers of mutuals, one-ways, 
and indifferences were observed to be so stable. The numbers of mutuals 
were 15 in September, 13 in November, 14 in January, and 16 in May; the 
one-ways were 45, 46, 47, and 43; while the indifferences were 240, 241, 239, 
and 241. What was perhaps surprising was the fact that the "initial" or 
September distribution (15-45-240) conformed exactly to the limiting values. 


Girl and Boy Subgroups 


Taba [3] notes a sex cleava 


ge and consequently the data on transitions 
were examined separately for 


boys and girls. The numbers of transitions 
TABLE 5 


Distributions of Types of Transitions in Two-Person Choice Structures, 


Boy to Boy, Girl to Girl and Cross Sex by Five Pairs of Time Points 


Type of Sept. to Nov. Nov. to Jan. Sept. to Jan. Jan. to May Sept. to May 
Е NR 
B to G to X B to G to X B to G to X B to G to X B X 
Transition B G Sex B G Sex B G Sex B G S " ұғы 


ех в G Sex 
Hutual to... 


Mutual 2 5 0 3 


[7 0 1 5 0 2 4 0 3 0 
Indiff. 1 2 0 3 3 0 2 0 0 1 0 2 0 
5 
1-мау to... 
Mutual 4 1 0 0 | 2 3 1 | 2 2 | 
$ате 2 15 0 8 1 2 10 2 3 5 2 : 
Opposite 1 | 0 1 2 0 0 0 1 0 | 0 | - ) 
Indi ff, 3 11 7 4 13 1 6 - 4 
15 3 6 11 8 
5 18 6 
Indiff. to... 
Mutual 1 0 0 0 0 0 2 0 2 | | 
E | 4 3 
| л 12 3 | 11 14 5 12 12 5 11 1 4 14 
п 5 ІЗ 70 134 16 72 12 | 
7 5 69 125 16 76 123 15 65 125 
Sums 36 120 144 36 120 


AE 


M 
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were counted for the 36 pair-relations among the 9 boys and for the 120 
relations among the 25 girls. These appear in Table 5. Due to the fact that 
more than 85 percent of the 144 сгоѕѕ-ѕех pairs were consistently indiffer- 
ences, the choice behavior of the whole class can almost be considered as the 
sum of two disjoint subgroups. 

Tests for constancy of transition probabilities from a two-month or 
four-month gap to the next and for second- versus first-order dependency 
were performed for the girl-to-girl choices. No departure from constancy 
nor from a first-order chain were detected; however, the number of cases 1s 
small. Tables 6 and 7, which parallel Tables 2 and 3 above, are given for 


completeness. 


TABLE 6 


Second Order Transitions in Girls! Two-Person Choice Structures 
from September-November to January 


Structure in Structure in January 
ыыы == c en 


ктк. т c 
November Mutual 1-мау Indifference Sums 


September 


mutual 3 0 2 5 
Mutual: 1-мау 2 0 0 ы 2 
іпді ҒҒегепсе 0 0 2 2 
mutual 0 0 1 1 
Mutual: l-way 3 7 6 16 
indifference 0 3 8 11 
mutual ] 0 0 1 
Mutual: 1-мау 1 4 7 12 
indi fference 0 8 62 70 


10 22 87 120 


TABLE 7 


Second Order Transitions in Girls! Two-Person Choice Structures 
from September-January to May 


MRNA noe ar S ананна 


Structure in Structure in May 

ОРНЫ Бане Inr m 

September January Mutual 1-мау Indi ҒҒегепсе Sums 

mutual 3 1 | 5 

Mutual: 1-мау 0 0 0 0 

indifference 0 0 4 h 

mutual 1 2 0 3 

1-мау: 1-мау 0 ц 6 10 

indi fference 0 3 12 15 

mutual 0 2 0 2 

Indi fference: 1-мау 3 h 5 12 

indifference р 8 60 69 

ы 8 2h 88 120 
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Of interest is the similarity of the boys and girls with respect to the 
relative frequencies of the transitions. To show this, tables such as the fol- 
lowing were examined for the combined (September to Ni 


ovember with 
November to January) two-month transitions. 


Starting from a Mutual Sub-group 
and ending two months 
laterat... Boys Girls Sums 
Mutual 5 9 14 
l-way 3 2 5 х = .90 
Indifference 4 5 9 
Sums 12 16 28 


starting from one-way, 1.21 ; Starting from indifference, 
month gap they were: Starting from mutual, 
1.04; and starting from indifference, 4.06. These 


two subgroups. 


Summary 
After proposing that interpersonal choice behavior viewed through time 
some data from an eighth-grade class 
examined. Within th 


of exploratory choosi: 
lepa St-order Markov chain model achieved st; tis int 
significance among the within-sex choices, паеш! 
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PATTERN VARIANTS ON А SQUARE FIELD 


5. J. PROKHOYNIK* 
THE UNIVERSITY OF NEW SOUTH WALES 


A quantitative approach to the psychology of pattern recognition re- 
quise knowledge of the number of possible variants of any particular pattern. 
"he general solution for the number [p/7m*] of pattern variants that p counters 


ean form on a square network of m? positions is obtained by elementary 
group theory. The exact solution is given in terms of the different types 
(symmetric, asymmetric, etc.) of patterns possible and an approximate 
formula for the total number of patterns is also developed. 


Given a number p of identical counters nnd a square network of m 


positions to place them on, iV ìs possible to form a finite number of configura- 
tions given by wd However, if only different patterns are allowed, the 


number is reduced considerably, since certain sets of configurations differ 
from one another only by rotation and/or reflection of the field; each such 
set represents a single pattern. The number of elements in the set will vary 
according to the type of symmetry or asymmetry of the particular pattern. 
Solution for the number of pattern variants requires differentiation among 
four types of configurations as well as a number of subtypes. 

For simple cases, particular solutions can be obtained, though labori- 
ously, by means of elementary arithmetic and geometry. Application of 
group theory enables a general solution applicable in all cases. The solution 
is extended with comparative ease to the ease where the pattern eonsists 
not of identical counters but of two or more different types. This requires 
only the application of a permutation factor which will generally differ for 
each of the four types of configurations. The problem has found (in fact 
arose from) practical application in certain psychological experiments, work 
by S. Kamocki, not yet published. It may also be useful, using three-dimen- 
sional networks, in physico-chemical problems involving patterns of dis- 
crete particles, for example the pattern of amino acids in the protem molecule 
or the configurations of other molecules or physical structures. 


Problem in Terms of Postulates 

identical counters on a square network 
where configurations which differ from 
are considered identical. 


The number of ways of placing р 
of т? positions is defined to be [p/m], wl 
one another by a rotation and/or reflection 


«Тһе author is indebted to Mr. J. Sandiford for his invaluable assistance on the theo- 
retical aspects of the problem and to Mr. J. L. Griffith for his helpful criticism. 
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We may establish a coordinate system on the network, with the axes 
dividing the squares into four quadrants. Taking т = 90 for even networks 
and т = 2n + 1 for odd networks (n — 1,2, 3, +++), then the range of 
2, y will be 


(== =Í o de 1) 
2 * 2 а 
Define the operation R аз а counterclockwise ro 
Thus R(z, y) = (-y, +x), R* = I. Define the operation S as a reflection in 
the middle vertical line, i.e., the y-axis, Thus S(z, y) = (—2,9), Si 7. The 


number of systems which are not equivalent under S, В or any combination 
of S and R is required. 


tation through 90°, 


Properties of the Group {R, 8} 


Denote {R, S}, a group whose elements are R, 8 and the identity ele- 
ment J, as the group Gs which is of order 8. Thus G, = (2, S} = 1, В, RA 
R58, SR, SR', SR?, wh 


operation SR repre- 
sents the operation R followed by the operation 5, 


that SR = R°g is a reflection 
RS is a reflection on the other 
the horizontal axis. Hence the 
as three subgroups of order 4, 


on the diagonal of Positive gradient; SR? = 
diagonal; and SR? = RS is a reflection on 
Group б) is complete as defined above. G, h 
viz., the cyclic group 
Ga = {R} = Т, В, R°, р, 
ба = {Е?, Б} =I, S, SR?, nS 
Gy = (2°, SR} = 1, SR, j SR’; 
and the five cyclic subgroups of order 


gi {R°}, С» = {5}, б, = 


Il 


2 denoted by 
{SR*}, Са = {SR}, Gas = (Sm). 


Types of Configurations Possible 
Define symmetric, Semi-symmetric, 


tion in the group G, . Th 
positions has been define 
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monstrated by Figure 1 and is unique, each one representing also a unique 
pattern. [Gs] will designate the number of patterns of this type in any par- 
ticular case. 

Semi-symmetric configurations (Figures 2, 3, and 4) are those invariant 
under the subgroups of order 4, but not under Gs . It is easily shown (А) 
that there are two, and only two, configurations of this type, (each invariant 
under the same subgroup of order 4) based on the same pattern, i.e., a semi- 
symmetric pattern. Thus the number of semi-symmetric patterns is [G,] and 
the number of semi-symmetric configurations is 2[G,]. It will be found con- 
venient to denote the number of patterns in the three subgroups of order 4 
by [Gi], [Сг], [Саз] pertaining to pattern types exemplified by Figures 2, 3, 
and 4, respectively. 


SS Se 


a FIGURE 1 FIGURE 2 
А Symmetric Gg Pattern А Seni-synnetric 0,1 Pattern 


М FIGURE Ш 


FIGURE 3 


А Semi-symnetric Oo Pattern A Seni-synnetric 9,5 Pattern 
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Anti-symmetric configurations (Figures 5, 6, 7, 8, and 9) are those in- 
variant under the subgroups of order 2, but not under any previous groups. 
There will be four, and only four, such configurations based on the same 
pattern, 1.е., an anti-symmetrie pattern (A). Thus the number of anti-sym- 
metric patterns is [G], and the number of anti-symmetrie configurations is 
4[G,]. The number of patterns due to the five subgroups of order 2 will be 
denoted by [бы], [Gas], [Gas], [Gos], 16-41 pertaining to the pattern types ex- 
emplified by Figures 5, 6, 7, 8, and 9, respectively. 

Asymmetric configurations (e.g., Figure 10) can be considered those in- 
variant only under 7, or, alternatively, those which do not belong to the 
2-, 4-, or 8-огдег groups of G. Every operation of the group G will yield à 


e eight configurations for each asymmetric 


different configuration; there ar 
ase [p/m] will be given by 


pattern. The number of such patterns [G,] for the с 


(7°) — ea - 2164 — 404 
ai= Т” 


Preliminary Theorems 


The following theorems, which follow directly from the postulates above, 


are useful in the solution of the problem. 


[s] e] 


an m? network there is a corresponding 


THEOREM 1. 


since for every pattern of p counters on 
pattern of (т? — p) blanks and vice versa. 
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TuronEM П. For every configuration invariant under Goo = {S} there 18 
an equivalent one invariant under Gaa = (SR ] and vice versa (A). Figures 
6 and 7 exemplify a pair of such configurations. 

Hence 


[Goo] = (6.31 = [Gx + G^]. 
From similar considerations, (e.g., Figures 8 and 9) 


[©] - [G25] = [Ga + Gas]. 


Тнконем Ш. In any odd network the пи 


mber of patterns for [Gs], [С], 
and [Су] will be the same forp = 


2r + 1 (counters) as it is for р = 2r (A). 
Method 


It is clear that different considerations apply for odd and even cases, 
both for networks and for number of points forming the pattern. Further, 
symmetrical (Gs) patterns of p points over an m° network are possible only 
if p is a multiple of 4 and, in the case when m is odd, also if p — 1isa multi- 
ple of 4. Hence solutions of [p/m] are given for m = 2n, 2n + 1, and р = 4r, 
ár + 1, where r, n = 0, 1,2, «ss. 

Now any part of a solution for 4r or 4r + 1 coun 
only multiples of 27 is applicable for all even 
cases respectively, on the same network. Thi 


ters which involves 
(p = 2s) or all odd (p = 2s + 1) 
5 leads to the following theorem. 
THEOREM IV. If we substitute 2r = s in the solutions of [4r/m?] which in- 
volve only multiples of 2r, then we obtain the solutions of [2s/m^] when s = 1,3 


8 between [(4r + 1)/т and [(2s + 1)/m/]. 


ationship is necessary. It is also sufficient 


9, +++. А similar relationship applie. 


Proor. It is clear that the rel. 


ons for р = 2s,2s+1(s= 13 5... 
starred (*) set of the 


[Gaz] — [С]; [Gaz + Ga] = [Gas + б]. 

In deriving the general solutions, a set of т 
particular group or subgroup of G can b 
subsets of single points, pairs, quartets, 
group or subgroup. The solution then 


Points invariant under a 
е considered as being Composed of 
and octets invariant under the same 
Consists of all Combinations of such 
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subsets invariant under the particular group considered. For instance 8-point 
invariant sets occur only in the symmetric (Gs) patterns, being quartets of 
pairs of points symmetric about a diagonal or axis. Thus the solution for 
[Gs] consists of (all possible combinations of) quartets of diagonal or axial 
points and/or the symmetric octets as described above. 

Numerical evaluations of the general solutions for a number of patterns 
and networks are given in Table 1. The results agree with arithmetical 
enumeration methods which are possible though laborious for small n, 7. 


General Solutions 
(i) Solution for [4r/ (2п)?]. 


1-0) ( * Xe 7) ^ Jeep) 
кезе (06 9) «ео, 


m. 9) © ur ap 1), 
ағы = AC) 
вы = (7) в} 
шт - 4629079 
+ P on Ally a if Tc -i)- 
(быр : [6] = (e) = [Gel = 25. 
ena ci) + BNE =) 
Ы (ARS) ЕЕ cele] — шы). 


е = (0) - 162 264 - ТІ 


that is, 
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(ii) Solution for Ес ; 1]. 


(быр: Nil. 
[G4]* : Nil. 
[Gs]* : [Ga], [Goo + 6, з], Nil. 


б» + бы = APY») + (mar - =) 
«GE 22) +--+} 
GP = бы ) Б ме). 


(iii) Solution for [ 


4r | 
(Qn + 1} 
еее), 
[G] : (Gu) = {+ "| if в}, 


еы e [ж Jan 


: [Ga] = a{("" ақ. [Gl — 216, 1, 
(6. + Gos] + [Gos «+ вы] = (2 ма i: +) 
unde Тез stag ag 


[G,]* = (е) - ТЕТЕ ". 


(iv) Solution for] fet 4 +1 1 
Qn 1? 
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[G] 

[6] : бы, as for (iii) —Theorem III. 
(СЫР + [Gas]*- 

[G;]* : [Gai], 
[Goo + Gos] + [Ges + G25] 


= M + к + y " (ж + у + ") 
1 2r 3 2r — 1 


>> tree вазы 
ar = "(ая D) - б] — 2184 - це). 


Ratio of Symmetric to Total Number of Patterns 


Table 1 gives a selection of numerical evaluations of [Gs], [G4], [G], [G], 
and the total number of patterns for networks up to 6° positions. The total 


2 


number of configurations М each case is also listed. It is clear from Table 


1 that the total number of symmetrical patterns [Gs] + [G4] + 162] is only 
а small ratio of all patterns possible for а particular case, and further that 
this ratio decreases with increasing т, p according to а square root law. 
The latter also follows more rigorously by examining the nature of the 
particular and general solutions outlined above, whence it is seen that 


1452 
{СЛ + [G4] + 162) is, in the main, а function of т and hence of m” 
эр 


2 

а. (ФА А 2 " 

and lower powers of m, while ( ) is а function of m?” and lower powers. 
p 


Further it is also easily shown that 


(5) «46 =) 


Thus it would appear, both from the trend apparent in Table 1 and from 


the nature of the solution formulas, that (165) + [Gs] + @} approximates 


2 + 
ү, rather more for cases where the symmetry is more pronounced (as 
? 


for odd networks in general) and rather less for the other cases (as for odd 


positions in even networks, where, for example, there is no contribution from 
[б], [G4], [Gai], and [С + бз). 

Тһе small ratio of symmetri 
generalizations of practical consequence. 


c to total number of patterns leads to two 
First, in any physical problem in- 
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TABLE 1 


Numerical Evaluations íor Small m 


Total number Total number of 
of patterns configurations 
le: | [с, | 


Yy 
Бе (9% 


3 
"9 
үтте. 
о 
со 
Кі 
— 
2 
> 
б 


z | 0 0 1 0 1 + 
5 5 к 1 1 0 2 6 
34 yx 20 : ‘ а : A 
3 4 2 0 11 10 23 126 
4 ls Fo 0 2 . : di 
i 2 0 2 9 10 21 120 
i 3 б 0 14 63 77 560 
5 4 2 5 38 207 252 1,820 
2 © 0 0 42 525 567 4,368 
4 6 0 6 9] 954 1,051 8,008 
4 7 0 0 70 1,395 1,465 11,440 
4 8 2 10 112 1, 550 1, 674 12, 870 
5 1 1 0 4 1 6 25 
5 2 0 4 17 28 49 300 
5 3 0 4 57 258 319 2,300 
5 4 4 7 152 1,503 1, 666 12, 650 
5 5 4 7 328 6, 475 6, 814 53, 130 
5 6 0 20 645 21,810 22, 475 177, 100 
5 7 0 20 1, 085 59, 540 60, 645 480, 700 
5 8 7 28 1, 712 134, 333 136, 080 1, 081, 575 
5 9 7 28 2. 372 254, 178 256, 585 2, 042, 975 
6 1 0 0 3 3 6 36 
6 2 0 3 24 66 9з 630 
6 3 0 0 55 865 920 7, 140 
6 4 3 15 264 7; 227 7,509 58, 905 
6 5 0 0 468 46, 890 47,358 376, 992 
6 © 0 28 1,698 242,618 244,344 1,947,792 
6 7 0 0 2,460 1,042,230 1,044,690 8,347,680 
6 8 6 87 7,062 3,778,989 3, 786, 144 30, 260, 340 
6 9 0 0 8, 960 11,763,430 — 11,772, 399 94 x 106 
T 108 21,468 3 31.762, 596 31, 784, 172 254 x 106 
EN 0 0 24, 024 75, 088, 650 75, 112, 674 


volving the elucidation of 
advantageous to seel any evidence of symme 
symmetry is displayed, not only is the number of patterns possible reduced 
significantly, but also these can be listed and checked systematically under 
the groups and Subgroups of Gs , Gi , and С. 


total number of patterns [p/m?] 


the pattern of а structure, it woul 


is of the order of Қ” 
р 


greater than this number. Hence as a Second approximation 


| 
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El - ("Ja а, 


where є is а small fraction and e — 0 as m, p 9. 
It may be shown that а good approximation for eis given by 


Ё | = [б] + [Gs] + [Gil + [Gs], 


m 


for if 


where [G,], [Gs] are very small compared to [G:] (especially for large m, р), 


Шеп 
$- | Wn), em 


р 


2 2 
- (m eA) 
p p 
This formula gives a good approximation for the total number of patterns 
as may be seen from Table 2, but it is, of course, no substitute for the general 


solution when an exact enumeration of the different types of patterns is 
required. 


APPENDIX 


Proof that every semi-symmetric pattern has two and only two configurations 
Consider a configuration C invariant under Ga but not under Gs, 1.е., 


C = RC = RC = EC. 


Now consider the configuration 


Е = 80; 
Шеп 
RF = RSC = SEC = SC =F, 
РЕ = R(RF) = F, 
апа 


RF == R(R’F) = Е, 
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TABLE 2 

Error of Approximation Formula for Small m, p" 

Percentage 

m Р «é (actual) < (estimated) error іп (1+6) 
3 2 7/9 6/9 -6 
3 3 11/21 9/21 -6 
3 4 6/13 7/13 5 
4 2 2/5 4/11 -3 
4 3 1/10 1/6 6 
4 4 3/28 3/32 =2 
4 5 1/26 2/33 2 
4 6 1/20 1/22 -.5 
4 7 1/41 1/27 1.3 
4 8 1/25 1/28 = 
5 2 1/3 3/13 -8 
5 3 1/9 1/12 ЕГЕС 
5 4 1/19 1/28 -1.6 
5 5 1/38 1/58 -.9 
5 6 1/66 1/105 -.5 
5 7 1/108 1/173 2.3 
5 8 1/153 1/260 $53 
5 9 1/212 1/357 EA 
6 2 2/11 4/25 -2 
6 3 1/32 1/21 2 
6 4 1/50 1/61 -.4 
6 5 1/199 1/153 б) 
6 6 1/280 1/349 - .08 
6 t 1/848 1/722 .02 
6 8 1/1050 1/1375 -.02 
6 9 1/2627 1/2425 . 002 
6 10 1/2940 1/4000 - .009 
6 11 1/6250 1/6125 


В +0001 


Е , m? 
*e is estimated by / ч ) The actual value of ¢ is determined from 
p 


р 1 ут? 
=| = ( ) (+e), 
т? 8\р 

where [p/m?] is known. Both actual and estimated 


values are resented in 
where accuracy is not severely affected. The final c Y round figu, 


f gures, 
2 olumn Біуев percentage error і Н 
and thus in [p/m?] by use of е (estimated). Бе error in (1 


Fie 


therefore 


Р = ВР = RF = RP, 


included. If however Е = 
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fore there are no other configurations based on this pattern. Similar proofs 

apply to configurations invariant under Gy , баз . 

Proof that every anti-symmetric pattern has four and only four configurations 
Consider a configuration C; invariant under С», but not under ба, ба, 

ба, биз. be Cr = R?C, . Now consider the configuration C; = ВС, ; 


RC, = RC, = КЕС, = ВС, = С. 
If also C, = ВС, = б, then С, = ВС, = В?С, = ВС, , i.e., both C, , C; 
would be identical configurations invariant under Ga and hence previously 


included. 
If, however, C; = RC + C, then С, , C; are different configurations, 


based on the same pattern and both invariant under С», but not under any 
other group of higher order. It may be shown similarly that there exist two 


other configurations, viz. 
С, = SRC, and 6, = SC, , 


invariant under G;, but under no other group of higher order. 

Hence C, , C; , Сз, Cs are four different. configurations, all invariant 
under С, and based on the same pattern. Also the invariance of C; , C2 , 
С. , C, involves all the elements of 0661; therefore there can be no other con- 
figurations based on this pattern. Similar proofs apply to configurations in- 
variant under Gaz , Gas , ба) б. 

Proof of Theorem 11. Consider а configuration С, invariant under б», 
i.e., SC, = C, ; then there will always exist a different configuration С. = 
RC, ; but SR?C, = SR°C, = RSC, = RC, = С, , therefore C; is invariant 
under Gas = {SR}. Consider now a third configuration C, = RC: = RO; 
then SC, = SR°C, = В*5С, = РС, = C, , therefore C; is also invariant 
under G;; . Similarly it may be shown that C, = RC, is also invariant under 
б.з. These four configurations and their invariants exhaust all the elements 
of gsC, ; therefore the only configurations based on this pattern are two 
invariant under б and two under б.а. 

Proof of Theorem III. This theorem follows from the possibility of 
placing the odd counter in the unique central square (0, 0) of the network. 
The symmetry of configurations of 2r counters invariant under the 4- and 
8-order groups and of ба = [R^] is not affected, and yields а unique con- 
figuration of the same type with 2r + 1 counters. Conversely all patterns 
involving 2r + 1 counters in these groups must have the odd counter at 
(0, 0), the removal of which yields a unique configuration of the same type 


with 2r counters. 
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А battery of 18 tests of intermediate algebra and 20 reference tests was 
administered to two successive second-year algebra classes. Each battery was 
separately factor analyzed by Thurstone methods, and the two analyses were 
synthesized by the Tucker method. The five congruent factors obtained were 
identified as: Verbal Comprehension, Deductive Reasoning, Algebraic Ma- 
nipulative Skill, Number Ability, and Adaptability to a New Task. 


This study aims to explore intensively the small area of intermediate, 
or second-year, algebra by surveying the abilities employed in solving the 
variety of problems included in an intermediate algebra course. The data 
collected for the study provide an excellent opportunity to apply, and thus 
to test, Tucker’s method [14] for synthesis of factor analysis studies. 

The author’s hypotheses before conducting the study included: (i) for 
the solution of algebra problems important basic abilities are inductive and 
deductive reasoning, rote memory, number, verbal comprehension, and 
spatial visualization; (ii) a basic ability of algebraic manipulative skill is 
involved in intermediate algebra; (iii) fluency of expression is required in 
the solution of statement problems; (iv) all abilities necessary for the solu- 
tion of statement problems can be measured by multiple choice tests. 

To test these hypotheses a battery of 38 tests was administered to each 
of two successive second-year algebra classes. Of these tests 18 were algebra 
tests and 20 were reference variables. Separate tables of intercorrelations 
were completed for each of the two years, and each was factor analyzed by 
the Thurstone [11] complete centroid method. The separate factor analyses 
were combined by Tucker’s technique [14] for synthesis of factor analysis 
studies. The loadings of factors congruent to both analyses were determined. 
The axes were rotated in the congruent space to simple structure prior to 

factor interpretation. 


To determine the efficacy of multiple choice answers for statement 
tructed and included in the 


by Contract N6onr-270-20 of the Office of Naval Research and by Grant NSF G-642 of 
the National Science Foundation. The writer is indebted to Professors Harold Gulliksen 
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analysis. One of each pair used free answers and the other used multiple 
choice answers. The factor content of these pairs of similar tests was com- 
pared to determine whether or not different abilities were measured by the 
two methods of answering statement problems. 


Description of the Variables 
The following is a brief description of the 38 variables used in the study." 
In the parentheses following the description of each reference variable, 


numbered 19-38, is the name of the factor which the author expected to be 
identified by the test. 


1. Fundamental Operations: algebraic addition, subtraction, multiplication, and division 
of monomials and polynomials. 
Fractions: reducing, adding, subtracting, multiplying, and dividing algebraic fractions; 
multiplying mixed expressions; simplifying complex fractions. 
3. Factoring: all of the common types of factoring exercises. 
4. Quadratic Equations: some solvable by factoring, others requiring the quadratic formula. 
or method of completing the square, 


. Radicals: simplifying monomial radieals; fundamental operations of monomial and 
binomial radical expressions. 


2. 


Ezponents: problems based on the four basic laws of exponents; problems b. 


the meaning of the zero exponent, the negative exponent, and the fractional exponent. 
- Binomial Theorem: questions aimed to measure the student’s thorough knowledge of 
the binomial theorem. 


Progressions: questions testing the student's knowledge of arithmetic and geometric 

progressions, 

9. Use of Tables: reading tables of comm 
logarithms of trigonometric functions. 

10. Principles of Logarithms: questions aimed at measuring the student’s comprehension 
of logarithmic principles, the understanding of the definition of logarithm and its 
implications, and the four basic laws, 

11. Cartesian Graphs: questions on elementary analytic geometry of the straight line and 
the conic sections. 

12. Simultaneous Equations: pairs of equations, linear or quadratic. 

13. Informational Ability in Algebra: aimed at measuring the students familiarity with 
new verbal concepts introduced into mathematics at the intermediate algebra level, 

14. Theory of Quadratics: questions on the nature of the roots, the diseri 

lationship of the roots to the coefficients, 


iminant, the re- 
, and the parabolic graph of quadratic equations. 
15. Converting Statements to S ymbols—A : direct translation of 


words into algebraic symbols, 
16, Converting Statements to Symbols—B: like Test 15. 
17. Word Problems—B: writing the al 
the equations that would solve th 
18. Word Problems—A: like Test 17. 


19. Addition and Division: addition of three one-digit or two two-digit numbers; division 
of two-digit or three-digit numbers by a number between 2 and 12, inclusive, Prepared 
by Educational Testing Service [3]. (Number) 


Subtraction and Multiplication: subtraction of one-digit or two- 
“Copies of the tests have been de 
A 


ased on 


on logarithms, trigonometric functions, and 


gebraic expressions for the 


given verbal phrases or 
е stated problem. 


20. 


digit numbers from 
posited as Document number 6006 with the ADI 
duplication Service, Library of Congress, Washington 
citing the Document number and by remitting $15.00 
microfilm. Advance payment is required. Make checks 
Photoduplication Service, Library of Congress, 


Auxiliary Publications Project, Photo 
25, D. C. A copy may be secured by 
for photoprints, or $4.75 for 35 mm. 
or money orders payable to: Chief, 


| 


= 


* 


" — _ 


UU oU 


r^ 


28. 


29. 


30. 


31. 


32. 
33. 


34. 


35. 


36. 


37. 


38. 
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two-digit numbers; multiplication of two-digit or three-digit numbers by one-digit 
or two-digit numbers. Prepared by Educational Testing Service [3]. (Number) 


. Opposites: selecting from five words the one which is opposite in meaning to the given 


word. Similar to tests of Garrett [4], Harrell [6], Peterson [8], and Sisk [9]. (Verbal 
Comprehension) 


. Similes: patterned after Taylor's test [10]. The student finished incomplete similes 


with three different words or phrases. (Fluency of Expression) 


. Inventive Opposites: patterned after Thurstone’s test [12]. The student responded with 


two words opposite in meaning to the given word. The initial letters of the correct 
responses were given. (Fluency of Expression) 


. Lelter-Blank Sentences: sentences created by the subject, with the number of words 


equal to the given number of dashes and letters. Each word replacing a letter was 
required to begin with that letter. (Fluency of Expression) 


. Figure Analogies: similar to Army Air Force’s Figure Analogies [1]. (Deductive 


Reasoning) 


. Block Counting: the student counted in a pile of blocks, pictured in the test, the number 


of shaded blocks that touched the numbered blocks. Prepared by Educational Testing 
Service [3]. (Space) 

Letter Series: patterned after a Thurstone test [13]. The subject was required to indicate 
the fourth letter after the last letter in the given series. (Inductive Reasoning) 

Lelter Sets: similar to tests of the Thurstones [5, 13], Coombs [2], and Taylor [10]. The 
subject selected the one group of four letters that did not possess the characteristic 
common to the other four groups of letters in the item. (Inductive Reasoning) 
Family Trees: similar to the Thurstones’ Pedigrees [13]. The subject answered ques- 
tions concerning relatives of certain members on the given family tree chart. (Inductive 
Reasoning) 

Following Directions: similar to the Thurstones’ test [13], in which the subject per- 
formed simple tasks on the paper with a pencil, following easy directions (Deductive 
Reasoning) 

Similar figures: prepared by Educational Testing Service [3]. The subject selected 
the one of the four given figures which was a reversal. (Space) 

Tabular Completion: similar to Thurstone’s test [12]. The subject filled in missing data 
in tables of numerical data. (Deductive Reasoning) 

Word-Word: similar to Garrett’s tests [4]. A paired-associates test composed of common 
four-letter words. The subject was given four minutes to learn 20 pairs; immediately 
thereafter he was asked to complete each pair, when given one of the two words. (Rote 
Memory) 

Number-Number: similar to Thurstone’s test [12], like Test 34, but using pairs of two- 
digit numbers. (Rote Memory) 

Final Examination in Algebra: a test of The Choate School. The same test was used 
for the two years of the study. (Grades) 

Second Semester Grade in Algebra: for the student at Choate for the year of the study. 


(Grades) 
Final Grade in English: for the student at Choate for the year of the study. (Grades) 


Two Analyses and Their Synthesis 


The battery of tests was administered to all students in second-year 


algebra in 1950-51 and 1951-52 at The Choate School, an independent boys’ 
preparatory school in Wallingford, Connecticut. About 60 percent of these 
students had studied their first-year algebra at Choate; the others had begun 
their study of algebra in the school they attended prior to attending Choate. 
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The number of subjects included in the study was, by coincidence, 126 for 
each year. The tests were given in 19 testing periods, none longer than 40 
minutes. Although the reference tests were given without advance notice, the 
algebra tests were announced at least five days in advance. | 

Except for differences in teaching methods used by the five teachers in 
each of the two school years, the 11 divisions of second-year algebra were 
equally prepared for the algebra tests, in that all students received similar 
daily assignments. In 1950-51 the textbook used was: Welchons, A. M. and 
Krickenberger, W. R., Algebra Book Two; Ginn, 1949. In 1951—52 the text- 
book was: Snader, Daniel W., Algebra (Meaning and Mastery) Book Two; 
Winston, 1950. 

Instructions on the 18 algebra tests were brief, simple, and included the 
time limit, varying from 10 to 30 minutes. The least number of items in any 
test was 24. The author prepared all the items for these tests. The scores on 
the tests were converted into grades, which were a large part of the student’s 
final record in algebra for the year’s course. To achieve optimum performance 
from the students, the author informed them before the testing that final 
grades would depend upon their performance on these tests, 

Instructions on the reference tests were often quite elaborate, usually 
including examples and practice problems. Time limits varied from 4 to 20 
minutes. The brevity of the items enabled more to be administered in а 
shorter time than was true of the items on algebra tests. The number of 
items ranged from 26 to 100. Except for those five tests used with the per- 
mission of Educational Testing Service all the tes 


al Т t items were original with 
the author. For motivation the students were informed at the beginning of 


the study that their scores on these tests would be used for guidance purposes. 

Three of the reference variables were students’ grades as reported at 
the end of the year—Final Examination in Algebra, Second Semester Grade 
in Algebra, and Final Grade in English. 


The data for the first year of the 
study have been labeled Study A; the second year’ 


8 data carry the notation 
Study B. 
The scores for the individuals’ tests w 


ere intercorrelated separately for 
each year of the study. The two 


38 X 38 matrices of intercorrelations are 
given in Table 1. Each of the matrices was factored by the complete cen- 
troid method, using the highest entry in a column as an estimate of the 
communality at each stage of the factoring. Distributions of the final residuals 
after 12 factors were removed in each study, indicate that the common йот 
variance had been exhausted. The twelfth-factor residuals appear in Table 


A*, and distributions of these final residuals are given in Table В. The pro- 
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jections of the 38 test vectors upon the 12 orthogonal centroid reference 
vectors for Study A are listed in Table 2. Those for Study B are listed in 


Table 3. 
Following Tucker’s notation [14] the reference-factor matrices in Table 


2 and 3 are labeled Fjna and Ғ;мв, respectively. The subscripts, m and M, 
designate the reference factors. In synthesizing the two analyses two trans- 
formation matrices, Tmra and Targ , must be found so that only negligible 
differences in the matrices F;,4 and F;,, will result when 


F ira = Pal acá and Е;,в == Ё;мв1 wre . 


Only one of the 38 variables was not considered as an overlap test. 
Test 1, Fundamental Operations, was given without earlier preparation in 
Study A; whereas in Study B the students were given assigned work in this 
topie prior to the test. For this reason it was eliminated from the test of 


congruence. 
Table 4 lists the means and the standard deviations of the overlap 


TABLE h TABLE 5 


Means and Standard Deviations а 
tent Roots 


of the Overlap Variables — HÀ Ó— 
Test МА d: Өзі 243 Study А Study В 
ST: ЭНИНИН... ккал енен ШЕ =. дараа. АША 
: 20.12 21.00 4.11; .59 Axis A Axis 8 
17.11 қ „81 .60 кел== eam ELE 
ц а ыы : 78 ә p-i 14.h720 P-l 12.2673 
5 17.h8 20.52 5.10 3.20 p-2 2.361. P-2 2.3003 
6 20.33 20.90 1.68 5.76 p-3 1.1513 Р-3 2.0105 
7 17.13 20.1: 5.22 3.86 p-k 1.5515 Р-!; 1.2271 
8 9.60 11.25 3.79 3.90 р-5 1.2286 Р-5 0.9263 
9 11.33 8.01 5.21 1.33 р-6 0.7608 P-6 1.2936 
10 11.55 13.60 5.25 4.93 р-7 0.5706 Р-7 0.6472 
11 10.85 13.01 3.1.6 11.55 p-3 0.6358 Р-8 0.75119 
12 16.63 17.56 1.25 1.78 р-9 0.1106 Р-9 0.5348 
13 16.78 17.00 б.о! 6.53 р-10 0.3663 P-19 0.1859 
1h 11.02 10.67 3.68 3.26 р-11 0.51127 2-11 0.3932 
15 17.17 19.60 3.96 2.9) р-12 0.2553 Р-12 0.2563 
16 16.54 18.00 5.07 3.17 
17 12.90 14.60 1.12 3.93 
18 9.99 11.29 1.53 4.75 
19 37,57 38.56 7.17 8.57 
20 30.19 31.31 7.24 8.36 
2 ee 25.35 1.99 8.11, 
9.75 22.19 5.62 5.12 
^s 21.12 2.58 7.53 7.09 ТАВТЕ 6 
25 11 vw D wd Coefficients of Congruence 
26 25,82 26,92 1.90 4.93 (arranged in order of size) 
А е re al 
27 15.69 17-13 6.05 6.71 2 P 
o 48 кй Ja m a 
‚13 15. 2 3.2 . — me 
30 16.30 18.60 5,48 5.116 .9860 Е 
32 17.12 18.81 27.28 8.26 о pres 
33 22.31 25.27 10.53 9.79 . a "sis 
3h 16.33 17.87 7.26 8.25 .13 i 45240 
35 9.78 9.46 ,,86 4.05 э "228 
36 15.39 16.91 14,53 13.89 epe Т, 
37 73.92 16.1 9.39 8.33 ES 17 
38 75. 77.09 6.48 7.01 . 


= Mean score in Study А 
Mg = Mean score in Study B 


"a д " Standard Deviation of the scores 
in Study A 

9 1B = Standard Deviation of the scores 
in Study B 
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variables for each study. In order that а common unit of measurement 
would be used for each test in both studies, adjustments were made on the 
standard deviations, computed by the formulas 

dja 


[£71 Cig 
ar dnd e 8... 
Mace OO "15 ug ns 


Reference-factor matrices for the tests with adjusted units of measurement 
were computed by 


Рута = D;,F;,, and Fius = Р.вР;мв. 


The subscript J designates the tests after the adjustment of the standard 
deviations. Matrices Ру„л and Р ув are given in Tables C and D. 

The latent roots and latent vectors of the matrices F 
ЕумвРумв were computed on the IBM 701. The latent roots for both mat- 
rices are shown in Table 5. The latent vectors for Study А are 


given in mat- 
гіх A,,4 in Table E, and the latent vectors for Study B are given in matrix 
Aus in Table Р. 


7»4P sma and 


Three principal axes in each stud 
P-12), corresponding to the lowest la 
because it is necessary to exclude fro 


у (p-9, p-10, р-12, P-10, P-11, and 
tent roots in Table 5, were discarded 


m the congruent space those dimensions 
into which the overlap tests have small projections. As yet no rule has been 


formulated to determine which principal axes should be retained and which 


should be relegated to the noncongruent space. In this Study the author 
arbitrarily chose 0.5 as the critical value of the latent root. 


The Gramian matrix Н, is given in Table G. The latent roots of this 
matrix are the squares of Tucker’s coefficient of congruence, ¢, . 


Н, = GG', 
in which 


Q 
| 


P Tha (Foe AF rus) T app , 
where 


Toa = 


-1/2 = 
AmpaBa and Typ, = Aarpsfls ^, 


Вл is a diagonal matrix containing the latent roots of Ға А sma , and fs isa 
diagonal matrix containing the latent roots of P5, ;F sary . The latent vectors 
of H, are given in matrix A,4 in Table Н. 

Table 6 gives the values of $. 
study. As yet no criteria have been d 


, those 
bility of good tests might be consid 


TABLE 7 


Transforzatio: 


Rotated factors 


А B с в Е 
T 3 -0217 20318 -.0919 -0598 
ii — -.2379 „5020 -.0h26 .0510 „5615 
Ч -.21h6 „5058 „3699 „3226 -.572\ 
iv „3210 — -.1296  -.3033 +9286 19% 
v 1760 -.0243 BU .3012 +1063 
тї -.20680 = 231285 +1907 +0006 202 
vii +0903 24910 о-о — -.0h50 = 2U72 
viii .0721 .Ш32 -.1722 „0118 -.h261 
ix — .09%6 «237 «038 10028 -.1319 
x -.066  -.129?  -.0000  -.09]h 23331 
xi -.0h37 «068 -.5127 -.1007 «0565 
xii .0739 —— -.05h6 .1223 — -.1105 —— -.0233 
TABLE B 


Transformation Matrix -- Study В 
а _.——— 


Centroid Rotated factors 
factors x в С T Е 
т 47386 .0216 20397 -.0893 „076 
II — -.2108 7115 .0053 +1937 1150 
шш 3236 -c3h  -.095 2716 -0Ш9 
IV — -.0707 19h -.1326 -.2608 «1122 
У -.2733 --.2Ш72 „3249 — -.1919 +8580 
Mi +3767 -.0499 -.!л8 „3881 „232 
VII -.3836 «3112 .0219 — -.060 -+2498 
ин -196 Защ  -.055 -099 -1282 
№ dft 37 .1981 .2067 -.2112 
X -ilh9 — oxi .2005 -07 -:0868 
XI 041 — -00n г0873 -.060 .0983 
м  .0555  -.0579 .006h -.033 11199 
TABLE 11 
MN: quo ет 
Ф а b с а е 

^ +1490 512 121 «269 .26h 

B -.106 581 шот -.131 -.555 

с 46 -Ө% 2.79 -26 6 

2 ‚215 119  -.3h8 -.851 006 

E -.522 +309 271 -.258 76) 


TABLE 9 дау А 


or Loadings of Overlap Tests оп the 
ve Factors in the Congruent Space 
га * Fun rA 


2 E 
3 208 
4 193 
5 129 
3 
8 -.149 
9 159 
10 -.05 
u „02 
12 298 
x E 
Е zi 
-:03 
H zn 
19 cu 
a zx 
22 -@ 
23 2 
2h x 
25 g 
26 z Я 
27 2 
28 ‘a 
29 M 
30 AW 
E n 
32 205 
5 152 
эһ "20 
35 108 
36 E 
DNE ме “Мезі 


TABLE 10 Study B 


Factor Loadings of Overlap Tests on the 
Five Factors in the Congruent Space 


Fyr * Fay in 


warm тш сш шуен 
2 E 21 
DES de 
z 106 
2 .09 
1 2% 
Е 103 
13 

20 -.09 
A 19% 
ЕЕ -20 
33 sell 
25 елт 
15 103 
26 103 
27 2703 
18 a 
19 EC 
59 135 
-.22 

25 105 
23 2:04 
ж 123 
22 EJ 
z 136 
27 428 
28 A 
29 m 
m -.16 
oS 

32 08 
эз 52% 
x 5 
35 us 
26 102 
37 120 
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congruence. The five factors having coefficients of congruence above .80 
were retained in the congruent factor space; the remaining four were relegated 
to the noncongruent space. M" 

Tables 7 and 8 show the matrices T,,., and Ту,» which transform F Sod 


and Ёумв into two matrices in which the differences are negligible for the 
overlapping tests. 


T usa E Varad and а —-— Үм-ҺВ, , 


іп which 


"Y mr A 


= D dua and YMrB = (T w»5G^)(A,1)9;! , 


where D, is a diagonal matrix of the re 
average of the sums of the squares for c 
"Y mr A and 'Y Mr B Ы 


The matrix, P,,, = Ғ,һАТ,,л, shown in Table 9, represents the factor 
loadings of the overlap tests in Study A on the five factors Which are in the 


five-dimensional space congruent with Study B. The factor | 
overlap tests in Study В on the 


F ; w5T y, 5 in Table 10. 

Faetor loadings for each pair of factors in the matrice 
were plotted on orthogonal axes. After ten rotations the composite result of 
the transformation matrices, y,, , is given in Table 11. Tables 12 and 13 


show, respectively, Jas Bid Ts ‚ the transformation matrices for con- 
verting the factor loadings on the rotated oblique factor axes. 


ciprocal of the square root of the 
ntries in corresponding columns of 


в Рол әлі Fie 


TABLE 12 TABLE 13 
Transformation Matrice 


Factor Loadings on Orthogonal Axes to 
Factor Loadings on Rotated Oblique Axes 


msA Тизв 
i „2920 „3306 ‚1932 


„2507 „1335 


Transformation Matrices for Converting 
Factor Loadings on Orthogonal Axes to 


5 for Converting 
Factor Loadings on Rotated Oblique Axes 


D E 
I - 3318 +3614 +2034 


-2653 ‚1888 
НЫ -48 3750 .3090  -.330 0516 E ME NEP uu -.3600 
Hi 483 -1М48 2050 — ..089 — 6524 III 44225 1199 2.3889 -.5237 1108 
т 7 498 S A E MEM 020 0066 MS шош 
У «2269 3 10%  ..3 52 190 M NN pes 
E A әлан ш Шо 2 NC MNT +2100 
adio" M M M ШЕ чай 508 (йе -.0352 boys 
viii 2972 408 -.306 „2 „сув VIII — -656 — 2.0961 обол 0422: „1109 
% ie NS — GEL ли „эш IX 0718 — 2,963 |0609 -.3065 ^ ..2605 
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Tasa = Ym:aD, and Туһ = үм.вР, , 
where Ұша = Тау, з Әлі үл = Тый»: › 


and where D, is а diagonal matrix of reciprocal square roots of mean sums of 
squares for entries in corresponding columns of 7,,.4 and Yuan. _ 

Matrices F;,, and Ру.в , shown in Tables 14 and 15, contain the load- 
ings of the tests on the rotated factors obtained by 


Fissa = FynaT mea and Fup = ЁумвТм,в. 


Tables 16 and 17 show the correlations betw 
Tables J and К include the direction cosines for the noncongruent factors. 
Tables L and M list factor loadings on these seven factors of Study A and 


B, respectively. These were determined by methods detailed in Tucker's 
paper [14]. 


een the congruent factors. 


Interpretation of the Results 
Identification of the Factors 


е Reasoning; Factor О, 


ility; Faetor E, Adapt- 
ability to a New Task. АП tests with factor loadings of .30 or above in either 
study are listed for each of the factors. 


Factor A—Verbal Comprehension 


Test Study A Study B 
no. Algebra Tests Factor Loading Factor Loading 
13 Informational Ability Al .36 
15 Statements to Symbols—A .50 .46 
16 Statements to Symbols—B 42 .40 
17 Word Problems—B .38 .43 
18 Word Problems—A .34 .46 

Reference Tests 
21 Opposites .67 .64 
22 Vocabulary Completion .62 .63 
23 Similes .40 483 
94 Inventive Opposites .39 .39 
30 Family Trees .34 .40 
31 Directions .54 .45 
38 Final Grade in English .34 .36 


Factor A (Verbal Comprehension 
reference tests of vocabulary. Loadings 
tests that were largely composed of w 


) had its highest loading in the two 
above .30 appeared in all the reference 
ords except the paired-associates test. 


= 
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In the algebra tests, Factor А was evident only in the four statement problem 
tests and in the test of informational ability. 

Tests 15 and 16 involved the converting of verbal statements into 
algebraic symbols. Since the words were not unusual and were, in the author's 
opinion, thoroughly familiar to all the students, the large loading of these 
tests on Factor A suggest that this factor is somewhat contaminated with a 
reasoning element. Perhaps the factor could have been named ‘Verbal 


Reasoning." 


Factor B—Deductive Reasoning 


Test Study A Study B 
no. Algebra Tests Factor Loading Factor Loading 
6 Exponents .27 .38 
7 Binomial Theorem .34 .28 
8 Progressions .35 .36 
9 Use of Tables .45 44 
10 Principles of Logarithms 187 43 
п Cartesian Graphs 41 .36 
18 Informational Ability .33 184 
14 Theory of Quadraties .36 .38 
18 Word Problems—A .40 .32 
Reference Tests 
26 Figure Analogies AT .49 
27 Block Counting .38 .19 
32 Similar Figures .35 AT 
33 Tabular Completion .34 .35 
36 Final Examination in Algebra .50 .50 
37 Second Semester Algebra Grade .46 .46 


The nine algebra and six reference tests in which Factor B had loadings 
above .30 indicate that it is a factor of Deductive Reasoning. Tests 26 and 
33 are similar to tests which have been used earlier in factor studies to identify 
a factor of deductive reasoning, defined here as the ability to reason from the 
general to the specific—to apply a principle or a rule to а specific problem 
when the principle or rule has been previously given, explained, and under- 
stood by the subject. 


Factor C—Algebraic Manipulative Skill 


Test Study A Study B 
no. Algebra Tests Factor Loadings Factor Loadings 
2 Fractions .63 .57 
Б] Factoring .62 .54 
4 Quadratic Equations 47 .60 
5 Radicals .51 .40 
6 Exponents .30 .30 
12 Simultaneous Equations .24 .39 
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Factor C (Algebraic Manipulative Skill) (ce den lice PS 
tests composed of material that had been studied in e i y: geb = 
viewed in the intermediate algebra course before the tests were given. 
рш loadings occurred in those tests which required a maximum of E 
braie manipulation. The Final Examination in Algebra was in part inr ог 
of problems that required manipulation, but these problems satin e | 
only 32 percent of the total score. The other variables on which a larger 
loading may have been expected were Test 1, Fundamental Operations, 
which was a very easy test, and variable 37, Second Semester Grade in 
Algebra. The second semester in each year of the study was devoted to the 
topics of logarithms, numerical trigonometry, progressions, and the bi- 
nomial theorem. These topics involve very little algebraic manipul: 
comparison with the topics taught during the first semester. | 

That all the tests with significant loadings on Factor C represent topics 
taught in elementary algebra suggests the possibility that this factor is one 
of previous training. Evidence to the contrary exists in the factor loadings 
of Tests 15-18, which represent topics usually included in elementary algebra. 
Since these tests had very small loadings on Factor С, it is believed the ele- 
ment of previous training is not important in this factor. 

For all the tests having loadings above .30 on Factor C the directions 
were very concise, e.g., “simplify,” “evaluate,” “solve,” “perform the indi- 
cated operations,” etc. There is good evidence that the student was not re- 
quired to change set in the course of the test ; it is possible that this factor 
represents “an ability to maintain a fixed set." The author’s thorough study 
of the other tests leads him to believe that frequent changes of set were 
demanded in each. The possibility exists, too, that the factor may involve 
both the ability to maintain a fixed set and algebraic manipulative skill. 
Further testing and analysis is necessary to determine the identity of this 
factor. Its high correlation of .48 with Е 


actor B, as shown in Table 16, would 
suggest that some element of deductive reasoning may be involved in Factor C. 


ation in 


Factor D—Number Ability 


Test Study A Study B 
no. Reference Tests Factor Loading Factor Loading 
19 Addition and Division .82 „ЗІ 
20 Subtraction and Multiplication .80 .80 
25 Letter-Blank Sentences .30 .92 
29 Letter Sets .23 .34 
33 Tabular Completion 44 .50 


The very high loadings of Fa 
incorporated in the analysis to 
the name for this factor. Test 


ctor D on the refer 
aid in locating the nu 
33 also required a ¢ 


ence tests 19 and 20, 
mber factor, provided 
onsiderable amount of 


| 


Y 


y 
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simple arithmetic computation. Test 25, on the other hand, contained no 
numbers, requiring only the writing of a sentence of a given number of words. 
It is difficult to understand why the old concept of the factor of number 
ability played an important role in scoring high on this test; however, it 
might have been that a sense of number aided in composing sentences of the 
required number of words more rapidly. Test 29 contained no numbers and 
required no arithmetic computation. The subject was required to determine 
the characteristic that is common to four of the given five groups of four 
letters each. For this reason Factor D may best be described as a factor of 
number skill contaminated by some ability involving a fluency of ideas. 

The author's hypothesis that number ability would be found as a factor 
in much of the intermediate algebra is not substantiated by the loadings of 
Factor D in the algebra tests. 


Factor E—Adapltability to а New Task 


"Test Study А Study B 
no. Reference Tests Factor Loading Factor Loading 
23 Similes .30 425 
25 Letter-Blank Sentences 44 .48 
27 Block Counting .37 .44 
28 Letter Series .30 ‚31 
30 Family Trees .32 .29 
32 Similar Figures .30 .35 
34 Word-Word .39 .98 


AII the tests in the study which involved new tasks for the subject had 
loadings above .30 on Factor Е. None of the algebra tests had average load- 
ings on Factor E above .14 for the two studies. Similarly, all those reference 
tests which involved tasks that the subjects had previously experienced had 
small loadings on this factor. Factor E, then, seems to represent the ease 
with which the student can learn a new task. It is noteworthy that this 
factor clearly appeared even in connection with such relatively simple new 
tasks as those included in this test battery. To some extent, Factor E за 
factor of speed, but not the motor speed involved in performing simple 


manual tasks. 


Multiple Choice versus Free Answer 

Test 15, a test of converting statements to algebraic symbols, was con- 
structed with multiple-choice alternatives. Test 16 was constructed with 
problems very similar to those of Test 15, but having no alternatives pro- 
vided. The subject was required to give free answers. Similarly, Test 17 isa 
test of word problems, or statement problems, with multiple choice answers, 
Whereas Test 18, with the same kind of problems, requires free answers. 
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TABLE 18 
Comparison of Factor Loadings for Multiple Choice and Free Answer Tests 
Statements to Symbols Word Problems 

Multiple Free Multiple Free 

choice answer choice answer = 

Factor Study Test no. 15 Test no. 16 Test no. 17 Test no. 18 
A А .50 l2 .38 3k 
Verbal Comprehension B l6 40 49 .46 
B A +26 .28 mn „10 
Deductive Reasoning B 427 .26 .26 .32 
с А -.02 .02 EDI -.06 
Algebraic Manipulative Skill B -.15 +00 415 4.10 
D А 06 -.08 1h +03 
Nunber Ability B 10 .01 .05 .02 
Е А 09 .10 13 «0l 
Adaptability to a New Task В 15 B B 06 
" А 17.8 16.5 12.9 10.0 
Hears B 19.6 18.0 1.6 1133 

.0 51. Е ч 

Standard deviations x im 2 io rad 


The mean score of each multip 


le choice test for both Study A and Study 
B is higher than the mean score fo; 


r the corresponding free answer test; how- 
ever, the standard deviation of each multiple choice test is lower than that 
for the corresponding free answer test. The factor loadings on these four 
tests, Tests 15-18, as shown in Table 18, suggest little difference between 
the abilities needed to solve the multiple choice tests and those needed for 
solving the free answer tests. A study of the correlations of these two pairs 
of tests with the other 34 tests in the study, shown in Table 1, reveals this 
close similarity. The evidence tends to confirm the hypothesis: statement 
problems with multiple choice alternatives ¢ 


Је 1 ап be constructed to measure 
the same abilities as similar statement problems designed for free answers. 


Questions Remaining to be Answered 


This study would suggest the 


question: what serious consideration can 
be given to the factor loadin 


ings in a single factor analysis, by the Thurstone 
if in two studies almost identical in nature only 


, to analyze each part 
parts for congruence? 


; that Tucker's technique crams а maxi- 
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mum amount of congruence into the first few factors, and thereby creates 
faetors that are psychologically complex and somewhat obscure. This has 
added to the difficulty of identifying the factors with any large degree of 
assurance, and, too, % has added to the resulting high coefficients of correla- 


tion between the factors after rotation. 

If a Study C has been made for a third successive year, similar in nature 
to Studies А and B, and if this analysis had been synthesized with the syn- 
thesis of the first two years, would the number of congruent factors have 


been diminished below five? 

Factors of inductive reasoning, rote memory, spatial visualization, and 
fluency of expression—all expected by the author to appear in an analysis 
of intermediate algebra—failed to appear in the congruent factor space, 
unless they were undetected as a part of the five somewhat complex con- 


gruent factors. 


No attempt was made to rotate the noncongruent factors to a meaning- 


ful interpretation, and therefore no effort was made to analyze or to interpret 
the seven factors of Study A and the seven factors of Study B that did not 
fall in the space congruent to the two studies. The interpretation of these 


factors would seem to have value only in the analysis of the study for each 


year separately and not for the over-all two-year study. 
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REDUNDANCY IN TASK ASSIGNMENTS AND 
GROUP PERFORMANCE* 


Ћовевт B. Zasonc AND Упллам Н. SMOKE 


UNIVERSITY OF MICHIGAN 


The problem of combining abilities of group members to maximize the 
performance of the group as a whole is examined in terms of redundancy in 
task assignments. In particular, ways of distributing a given number of items 
of information among a given number of individuals to obtain the maximum 
probability of each item being recalled by at least one individual are studied. 
It is shown that there exists an optimal distribution scheme which is inde- 
pendent of the amount of material originally given, the size of the group, and 


individual differences in ability. 


'The problem of assessing determinants of group performance has been 
investigated from two different points of view. For the most part, studies 
in this area have been concerned with the effects of group variables, such as 
the presence of others [1, 2], cohesiveness [e.g., 8], leadership style [e.g., 5], 
and the like, on the performance of groups and of individuals working in 
groups. Recently, some attempts have been made to analyze the group 
produet by means of a combinatorial analysis of individual abilities. Lorge 
and Solomon [4] performed such an analysis in the area of group problem 
solving, and Hays and Bush [3] have used it in group learning. In principle, 
this latter approach is analogous to that of Moore and Shannon in what 
they called the “crummy relay problem" [6], which refers to constructing 
reliable cireuits out of unreliable relays. von Neumann [9] has shown that 
by using a number of components of limited unreliability a reliable machine 
may be constructed. Moore and Shannon have demonstrated that a reliable 
cireuit may be designed by using arbitrarily unreliable relays. The increase 
in circuit reliability is obtained essentially by increasing the redundancy 
among relays. It would seem that the study of group performance in terms 
of redundaney among the abilities of individual members would hold con- 
siderable promise. 

Consider, for example, a group of N individuals. Let Н items of informa- 
tion be given to these individuals. The object is to recover this information 
from the group as a whole after some interval of time. For the present purposes 
it is irrelevant which individual remembers a particular item, although the 
item should be remembered by somebody in the group. There is evidence in 


*This work was done under the sponsorship of the Behavioral Sciences Division, 
Air Force Office of Scientific Research, Contract AF 49(630)-33. 
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the area of individual recall to the effect that the proportion of items recalled 
is inversely related to the number of items originally assigned [10]. Conse- 
quently, the probability that а given item is remembered by a given indi- 
vidual is some inverse function of the number of items he was asked to learn. 
On the other hand, the probability that at least one individual of those 
assigned the given item remembers it increases with the number of indi- 
viduals assigned the item. The first consideration implies minimizing the 
number of items per individual, the second maximizing it. The problem then 
is to discover the optimal distribution of items among individuals. The group 


аз а whole is considered to remember an item when that item is remembered 
by at least one individual. 


Case I 
The following conditions are imposed on Case I. 

(a) The probability p(i, 7) that item $ is remembered by individual j is 
equal to the constant p, 0 < p < 1, or to zero according to whether or not 
item 2 is assigned to j. 

(b) Each individual is assigned the same number, h, of items. Thus 
È: pi, j) = hp. 


(с) Each item is assigned to an equal number, n, of individuals. Thus the 
probability, P, that a given item is recalled by at least one individual is 
given by 


(0 Pesulld-$-1—4ü-s, 


the same for all items. 

Under the conditions of Case I the problem is reduced to finding the 
assignment of items which generates the greatest value of P, i.e., findin 
values of p and n that will maximize P. If it is assumed that оце ів dealing 
not with number of items (a discrete measure) but with amount of кае. 
ог amount of information (continuous measures), P may be regarded as a 
differentiable function of p. The necessary condition for P to be maxi cs 
as a function of p is that dP/dp = 0. Since h is а differentiable functio ye 
and since by (b) and (c) Nh = Hn, it follows that n is also à differ we 
function of p. Thus, from (1) айнаны 


di dn 
2 жк = —(1-—:| -- т 
(2) ар ( p) d log (1 — p) в А 


Given p # 1, dP/dp = 0 only if the bracketed expression is zero, or 
М 


(3) 


Since n = МИН, 


dn 
Mop т) 
dp 18 ( p) j25- 0 


ығ” 
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dh h 
4 = = р) ->— = 
(4) dp 105 1 p) = 0. 
Thus far the function relating р to В has not been specified. However, 
the condition in (4) holds for any set of values p and h which satisfy the 
relation р = 1 — e^, for an arbitrary constant k. Thus 
P = 1 ы [1 MN (1 = үн = 1 -f 


Nk/H 


is a constant since all the terms in the expression are constants. Thus, if the 
probability of a given item being recalled by a given individual were given 
by p = 1 — е, all assignments would be equally good. Under these condi- 
tions what is lost in p by assigning more items to each individual is gained 
tly the probability that an item is remembered 


by increasing n, and consequen 
dependent of the assignment of items to the 


by the group as a whole is in 
group members. 

If, however, the relation between р and л is not given by p = 1 — € 
but by some other function p = f(A), then in general not all assignments will 
be equally good. In fact, on the basis of empirical data available in this area 
[10] it would appear that the function is of the form р = 67%”, for an em- 


pirical parameter k which depends on such factors as time, nature of the 
material, its organization, meaningfulness, or the like. This function fits data 


gathered by Oberly [7] with k = .10. Assuming р = 677” and finding dh/dp, 


/h 


(8) (1 — р) log (1 — p) — 2p log p = 0, 


which is satisfied approximately for p — .84. Figure 1 shows the relation 
between P and p for selected values of k with N/H = .01. 

Solving for №, h = Vlog .84/ ZĘ? = 42/k. Thus, the best assignment 
results when each individual is assigned .42/k items. In terms of p, the maxi- 
mum value of P obtains when each individual is given the number of items 
which would result in his forgetting about 16% of the material. Since the 
relationship between h, which denotes individual loads, and п, which reflects 
the amount of task-assignment redundancy, is known, the optimal amount 
of redundancy may be obtained. Thus, for Case I, n = (.42/k)(N/H), the 
optimal amount of task-assignment redundancy. 
te that the result obtained is entirely inde- 


It is rather interesting to no у 
pendent of the size of the group, N, and the number of items, H. Thus for 


any given number of individuals and any given number of items, .42/k items 
per person represents the optimal assignment. Of course, the optimal amount 


of redundancy and consequently the maximum value of P vary with the 
er this ratio the higher the maximum 


ratio of individuals to items. The largi ) ] 1 
possible value of P. Figure 2 represents the relationship between the maxi- 
mum values of P and the N/H ratio for some selected values of the parame- 


ter k. 


364 PSYCHOMETRIKA 


1.00 


FIGURE 1 
The Relationship Between P and p for Different V. 


alues of the Constant К. 


The above solution was obtained by assuming À to be а continuous 
variable. Given a group of N individuals and a collection of H discrete items, 
it will not be possible in general to assign the items to individuals in more 
than a small number of ways. Hence the assignments do not vary continu- 
ously. As a matter of fact, for given values of N and H only some of all the 
possible assignments satisfy the conditions (b) and (c). I 
strated that if D is the greatest common divisor of N and 


assignments of H items to № group members satisfying (b 
to D, gi i 


Case IT 


In Case II items of equal difficulties and 
capacities were considered. N 


ferences in recall will be examined. 
Again a set of conditions is imposed, 
(a) The probability, v, 3), 


individuals w 


ith equal recall 
ow the case where there exist 


individual dif- 


that item 7 is remembered by individual j, is 
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equal to the constant p, 0 < р < 1, or to zero according to whether or not 
item 7 is assigned to j. 

(b) The number, h; , of items assigned to individual j is such that p = 
€ ^^, where Б; is an empirical parameter obtained with respect to the 
individual j. 

(c) The items are so distributed that for each item 4 the probability that z 
is remembered by at least one individual is equal to the constant P’ = 1 — 
П, п – pG, 2), the same for all items. 

Thus, the above conditions imply that different individuals will be 
assigned different numbers of items, depending on their individual abilities to 
remember them. The conditions also imply that each item 2 is assigned to the 
same number of individuals n, or that the redundancy is equal for all items. 


Hence 
(6) PeI-IIL-sz$51e1--g* 
Again, the necessary condition that P' be a maximum as a function of pis 
that dP'/dp — 0. Thus 
dP' dn 
м > —-0--(1-»y| = - CM 
Ф di а-я [2 log (1 = p) - т =| 


If p = 1, then 


(8) (=p) log (1 — ри ао 


In terms of the above conditions the nu 


mber of assignments is е j 
and to J ;h; . Hence nH = Уһ, . Thu ыт 


в 
dn 1 аһ, 
dp = H > dp 


Wil 


From restriction (b), 


d б, ins 
Ф 1 = EL ығы 


dh; 
- = 201, бір = (2p log p) L du., 
ШЕ Шыны. 
'Thus 
h; = (25 1 dh; 
(2p log р) ab 
and 


h; = (2р1 йз. 
> Сова Жы 


--—"—" 


ROBERT B. ZAJONC AND WILLIAM H. SMOKE 367 


Therefore 
ж; PE жїр (8) DLE 177 
and 
(1 — p) log (1 р а= 1 р) 0801 0 50 п 
ар 2plgp , 
ог 
(9) (1 — p) log (1 — р) — 2р log p = 0. 


Note that individual differences do not influence the solution, as (9) is satis- 
fied for p = .84. However, the number of items, k; , to be assigned to the 
different individuals will depend on their recall abilities which are reflected 
in the constants k; . If the assignment of items to individuals satisfies (a) 


and (b), then for two individuals jı and jz 


exp (—№11,) = р = exp (Ei) 


or 
kihi = Б, 
Thus 
hi, = = h;, 
and in general 
hj. = > hi, 


for each member j, of the group. Since >. = nH, 
k; 1 
пн = Хе. =. My 


nH 
h;i, = (ш) 


Непсе 


1 E 
zi | 


and in general 


ao m = (0) 
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for each individual j. In this case individual differences are exploited by 
assigning fewer items to less able members and more items to the capable 
individuals. БЕ | 

и While е value р = .84 is optimal under the restrictions specified 
above, it remains to be determined whether a different solution is obtained 


by relaxing restriction (a) such that p(i, j), is no longer required to be a 
constant across individuals. 


Conclusions 


The solutions presented provide a standard а 
results may be compared. Empirical tests must со 
otherwise the restrictions imposed on the solutions 
Thus by careful experimental controls it can be 
determine the departures from the prediction. 

For instance, the conditions im 


gainst which empirical 
nform to the predictions, 
could not have been met. 
discovered what variables 


, and this effect can be evaluated 
readily by empirical tests. The difference between the values of the parameter 
k for individuals working together and for i 


ndividuals working alone would 
provide information on the effect of group interaction on individu 


al per- 
formance, and the difference between the corresponding values of P, in- 
formation concerning the effect of group interacti 


i on on group performance. 
It is noted that the model presented is not restricted to recall, and that 


it may, with slight modifications, be applied to oth 


› er behaviors, such аз 
learning, problem solving, or decision making. 
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A NOTE ON THE USE OF MOOD’S LIKELIHOOD RATIO 
TEST FOR ITEM ANALYSES INVOLVING 2 X 2 
TABLES WITH SMALL SAMPLES 


Irvine B. WEINER 
UNIVERSITY OF MICHIGAN* 


Mood's likelihood ratio test is generally considered an unreliable x? 
approximation in 2 X 2 contingency {а les containing expected cell frequen- 
cies less than five. Probability values were computed for 60 such tables as part 
of an item analysis for two 30-item alternate forms of a measure. The rank 
orders of the items, from best to worst differentiators, as determined sepa- 
rately by Mood’s test and by Fisher’s exact test correlated .97 for one form 
and .96 for the other. 


Item analyses are often carried out not to determine the correlations 
of dichotomously scored items with a total test score, but rather to select a 
given number of items which best differentiates between high and low scorers. 
This type of analysis is facilitated by the use of 2 X 2 contingency tables 
which compare for each item the score (one or zero) of extreme scorers on 


the total test. The writer recently administered to groups of 25 and 22 sub- 


jects two 30-item alternate forms of a measure which he wanted to condense 
d sampling, many of the 


into a single 30-item form. Because of the restricte 
expected cell frequencies in the resulting contingency tables were less than 
five. Since x^ approximations are generally not reliable with small cell fre- 
quencies [1], the probability values for the tables were computed directly 
by Fisher's exact test, and the 30 best items were identified accordingly. 
The probability values covered a wide range, from .008 to 1.00. DM 
Probability values for the tables were also estimated by Mood s likeli- 
hood ratio test ([2], pp. 257-281), which is distributed approximately as x 


with (r — 1)(s — 1) degrees of freedom for large samples. Mood states, 
however, that this large-sample approximation cannot be used without 


appreciable error when г and $ both equal two. To evaluate this possibility 
with the 2 X 2 tables involved here, а comparison was made between the 
rank order of the items (from best to worst) for each of the two forms as 
determined by Mood's test and as indicated by the exact test. For one form 
the rank-order correlation (Spearman's rho) achieved between these two 
methods was .97 and for the other it was .96. Thus, although аай 
for its use were not met, Mood's test gave а very dd о hos 
relative probability values for the items. This may indicate that where ranking 
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is the goal, as in the item analysis described, the x? approximation by Mood’s 
likelihood ratio test is an adequate statistical tool even with small samples. 
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BOOK REVIEWS 


Jane LoeviNGER. Objective Tests as Instruments of Psychological Theory. Psychological 
Reports, Monograph Supplement 9, 1957, 636-694. 


A brief journal review cannot possibly do justice to a 60-page monograph which 
is so tightly packed with highly significant ideas and original points of view. The mission 
of the monograph is to present the point of view that the time has come to dispose of 
classical concepts of validity and to replace them with a concept in keeping with modern 
science. In particular, the proposal is that such concepts as predictive, concurrent, and 
content validities be swept aside as mere handmaidens of a psychotechnology of dubious 
scientific status, The clean sweep would also include such off-brand varieties of validity 
as factorial validity. In their place, it is proposed, the only concept of validity which be 
retained is that of construct validity, a concept which is developed more fully in this 
an it has been developed elsewhere. It is argued that predictive, concurrent, 


monograph th 
are ad hoe concepts which have little to do with what the scientist 


and content validity 


does or what he needs to know. 
The concepts of predictive and concurrent validity represent a concept of science 


which was typical of the late nineteenth century and which was presented by Karl Pearson 
in his famous Grammar of Science. A great amount of psychometric work today would 
represent an approach to the discovery of knowledge compatible with that presented by 
Pearson, and yet, in the hands of philosophers and logicians Pearson's approach has not 
fared well. In a sense, the introduction of the term construct validity by the APA Com- 
mittee on Psychological Tests represents а major break with the classical correlational 
concepts of validity and an indication that psychometrics is recognizing the fundamental 
changes which have taken place in our conception of science since the days of Pearson. 
The central purpose of the monograph is to clarify the concept of construct validity and 
to begin to establish it as an idea of central importance in the development of a science 
of behavior. 

Jane Loevinger is attempting to initiate a revolution in thinking in the area of psycho- 
metrics so that the instruments that evolve will be instruments for scientific advance 
rather than gadgets in a technology. Objective tests, she believes, are to play a central 
role in the development of psychological theory. Tests must be based on a theory of test 
behavior which is to be related to a theory of behavior in nontest situations. The elements 
in a psychological theory, it appears, are variables, but it is not clear whether these are 
or are not to be defined by objective tests. РС ЕРЕ 

The monograph proposes that the concept of sue Fe А i: B ER i 


components, One of these is the substantive component w: 
the content of the items included in (and excluded from?) the test can be accounted for 


in terms of the trait believed to be measured and the context of measurement. Context 
includes psychological theory and, in particular, ‘the psychology of objective test be- 
havior.” " The writer suspects that what Loevinger means by psychological theory may 
be different from what he means by psychological theory, but the concept of psychological 
theory involved in the document under review is not developed to the point where such 
а comparison is possible. Within the field of psychometrics, test items e е. discussed 
as if their properties could be described in the same terms 88 other d ci iem per 
tone or a flash of light or an electrical discharge. The experimental psycho Г ni ely 
to consider the properties of test items as stimuli as thi f antecedent learn- 


e consequence о 
ing conditions and that the properties of the items must be mí 


easured in terms of those 
conditions, Thus a semantic count provides 2 rough measure of ty! pical exposure to par- 
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ticular words used in particular contexts, and this measure in turn can be used to predict 
some of the responses that will occur to the particular item when it is presented as a part 
of a test. Such a measure of semantic frequency is one dimension along which the substan- 
tive component of a test item can be fixed. Presumably there would be other components 
too. But how would the substantive characteristics of items be determined in the case of 
items from personality tests? How can one tie down the antecedent conditions which 
generate the particular stimulus values of the items? In the Loevinger monograph such 
problems are not elaborated and the reviewer is left with the uneasy feeling that Loevinger 
is concerned with intuitively derived psychological theory rather than that in which the 
constructs are hypothetical and include unobservables. This is no real criticism of the 
monograph but only points up the long and difficult road that lies ahead of the person 
who embarks on a program of integrating psychometries with psychological theory. 

The second major component of validity proposed in the monograph is the structural 
component, which is a much better defined concept than is the substantive component. 
The structural component refers to “the extent to which structural relations between 
test items parallel the structural relations of other manifestations of the trait being meas- 
ured.” Interpreted in terms of the language of systematic psychology, one might say that 
structural validity implies that relations between responses within test situations should 
parallel in some way the relations discovered between responses in nontest situations. 
The structural component refers to the extent to which a test is a manifestation of a law 
relating responses to responses. Such laws, commonly described аз R-R. laws, have been 
generally considered to be rather low-level laws; 


that is to say, the uses which they have 
are limited, and the boundary conditions for them can rarely be specified. Herein lies one 


of the major differences between psychometrics and experimental psychology. 
metrics, R-R laws are sought, while the experimental psychologist seeks S-R 1 
of the particular advantages which they possess. A thoroughgoing integration of psycho- 
metries and current psychological theory would require that psychometrics become a 
means of developing S-R laws, but perhaps it is much too early to hope for this develop- 


ment. Loevinger has made a beginning by identifying some of the problems which this 
integration involves. 


In psycho- 
aws because 


The monograph provides an overview of various structural models that have been 
used and the problems which they involve. Under this heading the author has included 
quantitative models, class models, and dynamic models. The material could be dry to 
read, but it is not. Novel ways of looking at the problems involved add a continuous source 
of freshness to the discussion, Indeed, the incidental comments alone make this mono- 


graph worth reading, but it contains much more than incidental material. 
Loevinger writes with a contagious enthusiasm for her subject. Whether the reader 
agrees or disagrees with the arguments presented, he will come away 


from the monograph 
with the feeling that he has lived through a refreshing and worthwhil 


e experience, 
University of Utah Повент M. W. Travers 


L'Analyse Factorielle et ses Applications. Paris: Centre National de la Recherche Scien- 
tifique, 1955. Pp. 431. 


For five days in July, 1955, 37 factor analysts and scholars interested in factor 
analysis from 12 countries, met in Paris to discuss papers on factor theory and its appli- 
cations. This book is the result of their meeting. 

The conference was conceived by Henri Laugier, well known as a physiologist and 
as a scientist with a strong interest in applied psychology. It was supported, in part, by 
a grant from the Rockefeller Foundation. Listed home countries for those actually present 
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were: France, 15; Great Britain, 5; Sweden, 3; Egypt, Germany, Israel, Switzerland, and 
the United States, 2 each; and Belgium, Canada, Spain, and Yugoslavia, 1 each. 

Among those who had been invited but who were not actually able to be present 
were two who were represented by papers: Sir Cyril Burt and L. L. 'Thurstone. Their 
messages of regret are on early pages of the report. In addition to his formal paper, Thurs- 
tone contributed a resumé of the Conference on Factor Studies held at Educational Test- 
ing Service, Princeton, New Jersey, in November, 1951. 

Papers were circulated in advance, and the meetings, with the help of translators, 
concentrated on discussion. Of 22 papers, 9 were originally in French, 13 in English. In 
the book all papers and summaries of the discussions are presented in French. The follow- 
ing list of papers is a good guide to the nature of its content. 

“Current problems and new methods in factor analysis,” L. L. Thurstone. 

“The Uppsala Symposium on psychological factor analysis, Uppsala, March 17-19, 
1953," E. A. Peel. 

“Dimensions of intellect," J. P. Guilford. 

“Factor analysis: methods and results," Sir Cyril Burt. 

“Те probléme général de la recherche et de la nature des facteurs en psycho-phys- 
iologie,” (The general problem of research on factors and of their nature in psycho-phys- 


iology), H. Pieron. 
"Relations of the n 
Hotelling. 
"Remarques sur |’ 
centroides," (Remarks on Hotelling’s 


methods), H. Pineau. 
“A statistical test for the stability of simple structure,” R. Bargmann. 


“Psychological meaning of factor analysis as a research method,” M. Yela. 
“Les facteurs psychologiques: quelques remarques sur leur nombre, leur identifica- 
tion, leur nature,” (Psychological factors: remarks on their number, identification, and 


nature), Miss С. Bernyer. 

“Facteurs observés et facteurs t 
retical factors in psychology), M. Reuchlin. 

“The radex approach to factor analysis," L. Guttman. 

“Utilisation du schéma de Spearman, dans le calcul des images," (The use of а 
Spearman formulation in the caleulation of images), J. М. Faverge. 

“Factor analysis and the problem of validity," H. J. Eysenck, m " 

“Emploi de l'analyse factorielle dans l'étude de la variabilité biologique," (The use 
of factor analysis in the study of biological variability), E. Schreider. TM à 

“Application de l'analyse factorielle à l'étude de la mortalité, (Application of factor 


analysis in the study of mortality), S. Ledermann. А . 
ет бов CH uw sur l'analyse factorielle linéaire et générale," (Theoretical 
remarks on general, linear factor analysis), G. Darmois. КЕЗ 
"Nouvelle méthode de statistique mathématique pour l'estimation des facteurs et 
de leur écart-type en analyse factorielle,” (А new method in mathematical statistics for 
d analysis), P. Delaporte. 


the estimati d their standard deviations in factor ; 
о г: d of R. B. Cattell, “А universal index for psychological factors,” 


гі БА д о abilities,” A. H. El Koussy. 00, 
“Doctor Sa: of ии tests: methodological considerations and some 
empirical findings,” T. Husen and S. Henrysson. 
t “The factor analysis of person corre 
© identify the factors,” E. A. Peel. | 
“Rapport de synthèse ? (Summary of the conference), M. uin, қ 
These papers and related discussions depict remarkably clearly the contemporary 


ewer multivariate statistical methods to factor analysis,” H. 


analyse factorielle de Hotelling et comparison avec les méthodes 
factor analysis and a comparison with centroid 


héoriques en psychologie,” (Observed and theo- 


Jations and the use of independent determiners 
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state of the science and art of factor analysis. Contributors to the colloquium represented 
a broad spectrum of views, both on theoretical issues and on practical applications of 
factor methods. s 

"There were differences in historical orientation. Thurstone's paper traced the origins 
of factor analysis to the contributions of Spearman, while Burt’s paper ascribed the first 
version of the two-factor theorem to Galton and the method of principal axes to Pearson. 
Burt stated that the fundamental formula used by himself and his associates prior to 
1925 was the “centroid equation," ultimately adopted by Thurstone as the basis of the 
centroid method. 

Controversies centering around the two-factor approach of Spearman as contrasted 
with the multifactor approach of Thurstone were not in evidence. However, the theo- 
retical position of Burt and his followers seemed to oceupy middle ground as far as em- 
pirical findings were concerned. A general factor was still important in mental test material 
but there was generous provision for group factors and special abilities, 

Considerable attention was given to mathematical issues, especially on questions 
relating to sampling. Hotelling pointed out that many of the original objectives of factor 
analysis (but not all) could be better reached by other multivariate methods. He treated 
factor analysis as а means of estimating the dimensionality of a population, pointing out 
that the greatest use of present factor procedures was to Suggest hypotheses susceptible 
of being proved more objectively by other methods. 


Darmois, mathematical statistician 
at the Sorbonne, pointed out a number of the probability considerations arising in factor 
analytic practices, while Delaporte presented a method of estimating the standard error 
of a factor loading. In a somewhat parallel development, Bargmann’s discussion of simple 
structure is of considerable interest. 

Several of the papers, notably those of Thurstone, Guilford, Burt, and Hotelling, 
presented points of view that would have been quite familiar to readers of Psychometrika 
in 1955. Thurstone was pleased that multiple factor analysis had been judged sufficiently 
useful to justify an international conference, and indicated some of its eventual scientific 
possibilities. Guilford summarized his findings of the preceding six years, using the general 
framework of the centroid method and orthogonal rotations to simple structure, and seek- 
ing factors of psychological significance in high aptitude personnel, especially in the areas 
of reasoning and creative intelligence. 

The volume is an important source of information on Guttman’s radex approach 
to factor analysis. In 1955, Guttman’s contributions to the classification of patterns of 
matrices of correlations were not yet well known. His views of the simplex and cireumplex 
were elaborated not only in his paper but also in numerous discussions, 

To this reviewer, one of the most surprising (and pleasing) papers was that of Pieron, 
who is well known to American psychologists, but hardly as a factor analyst. Yet such is 
Pieron's versatility that, on the basis of reading 150 papers, he presented a thoughtful 
resumé not only of factor theory but also of applications in areas of special interest to 
investigators of problems in physiological psychology. 

The future historian of factor analysis will find this book a source of considerable 
information on trends and unsolved problems in the mid-fifties. The contemporary factor 
analyst will find some material not readily available elsewhere, as well as a number of 
stimulating papers and discussions. 


Рр e". Pair Н. DuBors 
Washington University 
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Minutes of the 
1959 ANNUAL BUSINESS MEETING 
oí the 
PSYCHOMETRIC SOCIETY 


Meeting of the Psychometric Society was held in Cincinnati, Ohio, 


The regular Annual 
1959. President Frederic M. Lord called the meeting to order at 


on Tuesday, September 8, 
2:00 P. M. 


The minutes of the previous Annual Meeting were approved. 


lection of two new members of the Council of Directors, Dr. Jane 


On a ballot for the e 
Milholland were elected for a term of three years, ending in 1962. 


Loevinger and Dr. John E. 


Dr. Lloyd G. Humphreys reported for the Membership Committee. The Membership 
Committee nominated 45 persons as full members, 60 student members to be transferred to 
full membership, and 32 individuals as student members. 


It was moved, seconded, and passed that the following 45 persons be elected as full 


members: 


Sam Mayo 

Jum С. Nunnally 

Ellis B. Page 

George J. Palmer, Jr. 
Mohammed Y. Quereshi 
J. A. Radcliffe 

Frank Restle 

Lillian Capo de Rivero 


Frank B. Baker 
Donald B. Black 
Lyle E. Bourne 
Allen C. Busch 
Don J. Cosgrove 
John W. Cotton 
William F. Dossett 


S. David Farr 

David J. Fitch үу, S. Robinson 

Raymond W. Frankman Nathan Rosenberg 

Paul A. Games Richard A. Schindler . 

Clifford P. Hahn Charles F. Schumacher 

Dean Harper Elmer Louis Streuning 

Paul Heit Theresa G. Trittipoe 

Miguelina N. Hernandez Read D. Tuddenham 

Edwin P. Hollander David L. Wallace 

John G. Hurst Wimburn Wallace 

Thomas Owen Jacobs Sam C. Webb 

John E. S. de Jung Frederic D. Weinfeld 

William J. Laidlaw Warren W. Willingham 

Herbert C. Lansdell Francis A. Young 
Joseph Zeidner 


Russell S. MacArthur 

R. Daniel Malone 

It was moved, seconded, and passed that the 60 student members listed below be 
transferred to full membership: 


Thomas H. Burkhalter 


Nancy S, Anderson 
y Ralph McC. Chinn 
Norman Anderson Paul Ra mond Christensen 
David C. Beardslee au y ; 
х Norman Cliff 
Morton A. Bertin 
: Bernard Р. Cohen 
Nicholas A. Bond, Jr. 
: үу, C. Coppock 
Eugene A. Bouvier 
Mary Corcoran 


James G. Boyce 
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Lolafaye Соупе 
Elliott M, Cramer 
Kern \. Dickman 
Sidney Epstein 
Donald Estavan 
Gordon Fifer 

William J. Flynn, Jr. 
Roman Gawkoski 
Clifton W, Gray 
Isaiah Guttman 

Roy Hastman 

Robert Arthur Hehner 
Shinkuro Iwahara 
James L. Jacobson 
Thomas H. Jerdie 
Henry F. Kaiser 
Herbert D, Kimmel 
Rupert A. Klaus 
Henry E. Klugh 
Richard H. Lindemen 
Clifford E. Lunneborg 
Richard D. Mann 
Robert E. McClintock 


И was moved, 
student members. 


G. Ernest Anderson, Jr. 
Joan A, Barr 

Alan R. Bass 

Carl E. Bereiter 
Robert Besco 

Vincent М. Campbell 
Ronald Lynn Flaugher 
Jane Garrett 

Bert A. Goldman 
Marshall Greenberg 
David L. Grese 
Richard Earl Hilligoss 
Stephen M. Hunka 
Edward S. Johnson 
Francis В. Kapper 
Eric Klinger 


It was moved, 
their excellent work. 
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John F. Muldoon 
Warren T. Norman 
David B. Orr 

Lee Edward Paul 
William Prokasy, Jr. 
James C, Reed 
Charles V, Riche, Jr. 
Wiliam L. Robinson, Jr. 
Robert Sadacca 

K. Warner Schaie 
Morris Schnore 
Richard E. Schutz 
Robert W. Scollay 
Ahmed El Sensussi 
Roy G. Simpson 
Richard E, Stafford 
Robert Earl Stake 

E. Elizabeth Stewart 
J. G. Thomas 
Edward E, Ware 
Richard W. Willard 
Calvin E, Wright 
Joseph L, Zinnes 


seconded, and passed that the 32 persons named below be elected as 


George Walter Mayeske 
James J. McKeon 
Arthur Lee Miller 
Curtis В. Miller 
Jason Millman 
Richard Millward 
Alexander Т. Quenka 
Donald Clare Ross 
Malcolm 1. Slakter 
Timothy A, Smith 
Saul H. Sternberg 
Jack T. Tapp. 
Mehmet F, Turgut 
Bob J. Williams 
Paul L. Williams 
Morris Wolfe 


seconded, and passed that the Membership Committee be thanked for 


Dr. Wilfred A. Gibson reported for the Program Committee. Of twelve abstracts of 
papers submitted for consideration for presentation at the Annual Meeting, ten were accepted, 
One symposium was scheduled, The Committee cooperated with the Division 5 Program Com- 
mittee in finding a speaker for the joint dinner and in Scheduling the various technical and bus- 
iness sections. It was moved and seconded that the report be accepted with thanks. Motion 
passed, 


Dr. Harold Gulliksen reported for the Committee on th 


Psychometric Society. He reported general agreement u 
Celebration. It was moved and 
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It was moved and seconded that the Society appropriate a sum of up to $500.00 to be ex- 
„pended in connection with the 25th Anniversary celebration of the Society. Motion passed. 


It was moved and seconded that, in connection with the annual billings for dues, the 
Treasurer invite voluntary contributions from the membership for executing plans in connec- 
tion with the 25th Anniversary celebration, with $2.00 suggested as a typical contribution. 
Motion passed. 


Dr. Charles F. Wrigley pointed out that 1960 is not only the 25th Anniversary of the 
Psychometric Society, but also the 100th anniversary of Fechner's book and that it might be 
appropriate for the Psychometric Society to pay attention to this important anniversary. It was 
moved and seconded that this fact be brought to the attention of the committee arranging plans 
for the 25th Anniversary, together with the possibility of collaborating with the American 
Psychological Association in the matter of the Fechner centennial. Motion passed. 


The report of the Treasurer was presented by Dr. William B. Schrader. А copy is 
attached. The report was accepted with thanks, 


Dr. Ledyard R Tucker reported for the Auditing Committee, The Treasurer's books, 
bills received, cancelled checks, and bank statements for the fiscal year July 1, 1957 to June 
31, 1958 were inspected. All financial matters of the Society were found to be in good form 
and in accord with the report published in the December 1958 issue of Psychometrika, The 


report was accepted with thanks. 


On ballot, Dr. Philip H. DuBois was re-elected Secretary of the Psychometric Society 
for a term of three years, ending September 30, 1962. 


Dr. Wrigley reported for the Committee on Special Memberships. The Committee sug- 


gested that a category of "Foreign Affiliates" be established. Foreign affiliates would receive 
in the meetings of the Society, and 


Psychometrika, would have the privilege of participating 4 
would pay the same dues as student members; however, no endorsers would be required on 
applications and foreign affiliates would not have the right of holding office. Persons residing 


abroad who wish to become regular members could continue to do so, as at the present time. 
It was moved and seconded that steps be taken to establish this category of membership. Motion 


passed. 


also suggested that there be established a cat- 


i 5 ial Memberships 
The Committee on Specia! е! р ho have reached the age of 65 and who have 


egory of "Emeritus Members" for individuals wl à a 
paid full dues for 15 years. Such emeritus members would receive Psychometrika and would 


have full membership privileges, including the right to vote and to hold office. Бийде өч re 
would be $1.00 a year. It was moved and seconded that steps be taken to establish such a ca 


egory. Motion passed. 
It was accepted with thanks . 


The Secretary's report was presented by Dr. DuBois. 


Reporting for the Election Committee, Dr. Clyde Н. Co 
has been elected President of the Psychometric Society for 
October 1, 1959, 


ombs stated that Dr. Humphreys 
a period of one year, beginning, 


The meeting was adjourned at 5:00 P. M. 


Philip Н. DuBois 
: Secretary 
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Statement of Receipts and Disbursements for Fiscal Year 
Ended June 30, 1959 


RECEIPIS (Dues) 


Year Members Student Members 
1959 585 51 
1958 ho 
1957 12 1 
1956 4 - 
641 60 
$4,727.00 
Received with Dues for Corporation Publications 227.56 
Net overpayments 1.55 
Total Receipts $5,955.91 
DISBURSEMENTS 


Psychometric Corporation (90% of dues) $4,254.30 
Psychometric Corporation (Publications ) 227.56 
Stationery and Postage 


385.90 
Secretarial Services 214,88 
Bank Charges 1.09 
Те1ерһопе 16.18 
Refund 1.00 
Total Disbursements $5,103.91 
BALANCE 
Balance, June 30, 1958 $1,345.20 
Receipts, 1958-59 


1,955.91. 

EAT 
Disbursements, 1958-59 5,103.91 
$1,197.20 
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Balance, June 30, 1959 


PSYCHOMETRIC CORPORATION 


Statement of Receipts and Disbursements for Fiscal Year 
Ended June 30, 1959 


РЕ ЕЕ Е ашаытас Oo 
RECEIPIS 


Subscriptions (less agency discounts) 
Psychometric Society (90% of dues) 
Sale of Back Issues (less discounts) 2,490.71 
Sale of Monographs 5-8 (less discounts) 
Interest on Savings Accounts 
А Reprints 5 
Net overpayments 2.24 


„ Co—m 


Mailing List Use 
$11,135.20 
DISBURSEMENIS 
ғы Printing and Mailing Psychometrika 
Volume 25, No. 2, throug! %, No. 1 $1,696.h0 
Reprints 477.42 
Stipend of Managing Editor 
(7/1/58 - 6/30/59) 150.00 
Stipend of Assistant Editor 
(1/1/58 - 6/30/59) 500.00 
Stipend of Treasurer 
(7/1/58 - 6/30/59) 250.00 
Secretarial Services: Editorial Office 750.00 
х Secretarial Services: Business Office 296. 1h. 
Stationery anà Postage 218.88 
Mailing Back Issues ani Monographs 156.87 
Psychometric Society (Incorrect credit) 28.00 
Refund 1.00 
| Legal Fees 35.00 
Mailing List Use 5.07 
p Miscellaneous 27.00 
$11,192.38 
BALANCE AND RESERVES 
| Balance, June 30, 1958 $7,242.74 
Reserve Funds, June 50, 1958 
Englewood Savings and Loan Assn. 
Englewood, Colorado 3,500.00 
Metropolitan Savings and Loan Assn. 
Los Angeles, California 3,500.00 
| Total Я А $1, 2h2.7 
| Receipts, 1958-59 1h,135.20 
Sum 28,971.9 
| Disbursements 11,192.38 
Remainder 817,185.5 
Balance, June 50, 1959 $10,785.56 
Reserve Funis, June 30, 1959 
Englewood Savings ani Loan Assn. 
Englewood, Colorado 3,500.00 
Metropolitan Savings ani Loan Assn. 0959 
Los Angeles, California 3,500. 
Total, Balance and Reserve Funds %17,185.5 
OBLIGATIONS 
Estimated cost of Paychonetr ia, 
Vol. 2, Nos. 2- 
Printing and Mailing $6,000.00 
stipends (7/1/59 - 12/31/59) {20400 
Secretarial Services к 
1,300.00 
$10,185.56 
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ЕВВАТОМ 


In Cureton, Edward E., Note on Ф/Фиах - Psychometrika, 1959, 24, 89-92, 
the first sentence of paragraph 2, page 89, should read “It is well known that 


Ф can equal +1 only if p, = p, , and that it can equal —1 only if p; = ps 
([1], p. 324; [2], р. 342).” 
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